How to Use the Ranker Component

Summary

The Ranker tag in Label Studio allows users to rank items in a list or distribute them into predefined buckets. This tag is particularly useful for tasks like evaluating LLM responses, sorting search results, or categorizing items based on relevance. The Ranker tag works in conjunction with the List tag and can be customized with various parameters and styles to fit specific project needs.

Steps to Implement:

1. Ensure Proper Data Format:
   • Verify that your task data is correctly formatted. The List tag should contain an array of objects with required fields like id, and optional fields like title, body, or html.

               Example data format:

{
 "items":
  [
    {"id": "item1", "title": "Title 1", "body": "Body 1" },
    {"id": "item2", "title": "Title 2", "body": "Body 2" }
  ]
}

1. Correct Labeling Interface Configuration:
   • Ensure that the Ranker tag is correctly connected to the List tag using the toName attribute.
   • Example configuration:

<View>
    <List name="results" value="$items"
  title="Search Results" />
    <Ranker name="rank" toName="results">
      <Bucket name="best"
  title="Best results" />
      <Bucket name="ads"
  title="Paid results" />
    </Ranker>
</View>

1. Check for Browser Compatibility:
   • If the Ranker component is not rendering, try using a different browser. Some users have reported issues with non-Chromium based browsers like Safari and Firefox.
   • Example issue: GitHub Issue #995
2. Inspect Console for Errors:
   • Open the browser's developer console to check for any errors that might indicate issues with the Ranker component.
   • Common errors might include missing or undefined properties.
3. Update to Latest Version:
   • Ensure you are using the latest version of Label Studio, as updates often include bug fixes and improvements for components like Ranker.
   • Example: Users found that updating to the latest Docker image resolved issues with Ranker not loading.
4. Styling and Customization:
   • Use the <Style> tag to customize the appearance of columns and items within the Ranker component.
   • Example styling:

<Style>
    .htx-ranker-column { background:
  cornflowerblue; }
    .htx-ranker-item { background:
  lightgoldenrodyellow; }
</Style>

More resources

• Ranker Tag Documentation
• LLM Ranker Template
• Label Studio GitHub Repository
• Label Studio Blog: Five Large Language Models You Can Fine-Tune Today
• Label Studio Zero to One Tutorial