groundingauburn/hot_annotator

title: Fact Tag Annotation Interface emoji: 🏷️ colorFrom: indigo colorTo: blue sdk: docker # use "gradio" or "streamlit" if you use those SDKs

app_file: app.py # only for gradio/streamlit; not used with docker

pinned: false license: mit short_description: Annotating and editing fact tags for crowdsourcing. tags:

• —annotation
• —crowdsourcing
• —labeling
• —flask
• —a/b-testing ---

Fact Tag Annotation Interface

An interactive web application for annotating and editing fact tags in text, designed for crowdsourcing platforms like Amazon Mechanical Turk.

Features

🎨 Different Colors for Each Fact

• —Each fact tag (fact1, fact2, etc.) has a unique color for easy identification
• —Color legend shows all fact types and their corresponding colors
• —High contrast text automatically selected based on background color

🖱️ Interactive Editing

• —Click any fact tag to edit it in a modal interface
• —Real-time text editing with save/cancel/remove options
• —Visual feedback with animations and status indicators

📊 Progress Tracking

• —Live statistics showing total facts, edited count, and time spent
• —Visual progress bar
• —Change history tracking for each annotation session

💾 Data Collection

• —Automatic saving of all changes with timestamps
• —Session tracking for each annotator
• —JSON export of annotation results
• —Admin dashboard to review submissions

📱 Responsive Design

• —Works on desktop, tablet, and mobile devices
• —Professional appearance suitable for crowdsourcing platforms
• —Keyboard shortcuts for power users

Installation

1. 1.Clone or download this directory
2. 2.Install dependencies:

   pip install -r requirements.txt

1. 1.Run the application:

   python app.py

1. 1.Open your browser and go to http://localhost:5000

Usage

For Annotators

1. 1.Visit the main interface at http://localhost:5000
2. 2.Read the instructions and review the color legend
3. 3.Click on any colored fact tag to edit it
4. 4.Make your changes in the editing modal
5. 5.Add optional notes about your annotation process
6. 6.Submit when finished

For Administrators

1. 1.Visit the admin dashboard at http://localhost:5000/admin
2. 2.Review all submitted annotations
3. 3.View statistics and completion metrics
4. 4.Export data for analysis

Configuration

Custom Fact Colors

Edit the FACT_COLORS dictionary in app.py to customize colors:

FACT_COLORS = {
    'fact1': '#FF6B6B',  # Red
    'fact2': '#4ECDC4',  # Teal
    'fact3': '#45B7D1',  # Blue
    # Add more as needed
}

Sample Data

Update the SAMPLE_DATA dictionary in app.py with your own content:

SAMPLE_DATA = {
    "question": "Your question with <fact1>tagged facts</fact1>...",
    "answer": "Your answer with <fact1>corresponding tags</fact1>..."
}

API Endpoints

• —GET / - Main annotation interface
• —POST /api/save_annotation - Save annotation data
• —GET /api/get_task_data - Retrieve task data
• —GET /admin - Admin dashboard

Data Storage

Annotations are saved as JSON files in the annotations/ directory with the following structure:

{
  "session_id": "unique-session-id",
  "timestamp": "2023-12-07T10:30:00",
  "original_question": "...",
  "original_answer": "...",
  "modified_question": "...",
  "modified_answer": "...",
  "changes_made": [
    {
      "factId": "fact1",
      "action": "edit",
      "originalText": "old text",
      "newText": "new text",
      "timestamp": 1701947400000
    }
  ],
  "worker_notes": "Optional notes from annotator",
  "completion_time_seconds": 245,
  "total_facts": 6,
  "edited_facts": 3
}

Customization

Styling

• —Modify static/css/style.css for custom themes
• —Update color schemes and layouts
• —Add your organization's branding

Functionality

• —Edit static/js/annotation.js for custom behavior
• —Add new annotation features
• —Implement custom validation rules

Backend

• —Modify app.py for custom data handling
• —Add database integration
• —Implement user authentication

Integration with AMT

This application is designed to work with Amazon Mechanical Turk:

1. 1.Host the application on a public server
2. 2.Create an AMT HIT that loads your annotation interface
3. 3.Configure submission to capture the annotation data
4. 4.Use the admin dashboard to review and approve work

Security Notes

• —Change the secret_key in app.py for production use
• —Implement proper authentication for the admin interface
• —Use HTTPS in production environments
• —Validate and sanitize all user input

Browser Compatibility

• —Chrome 70+
• —Firefox 65+
• —Safari 12+
• —Edge 79+

License

This project is open source and available under the MIT License.