Team Ai
Apppublic

MasoomChoudhury/processor

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes
App README

Real-Time Batch Image Processing Pipeline

This project implements a real-time, event-driven batch image processing pipeline using Supabase for database, storage, and real-time capabilities. The system processes images through OCR and AI analysis using Google Gemini and Anthropic Claude.

Features

  • —Real-time image upload detection using Supabase Realtime
  • —Intelligent batching of image processing tasks
  • —OCR processing using Google Gemini
  • —Dual AI analysis using Google Gemini and Anthropic Claude
  • —Event-driven architecture with no polling
  • —Docker containerization for easy deployment

Prerequisites

  • —Python 3.10 or higher
  • —Supabase account with database and storage set up
  • —Google Gemini API key
  • —Anthropic Claude API key
  • —Docker (for deployment)

Setup

  1. 1.Database Setup
  2. 2.Run the SQL commands from db_setup.sql in your Supabase SQL Editor
  3. 3.This will create the necessary tables and enable real-time functionality
  1. 1.Environment Variables Create a .env file with the following variables:
   SUPABASE_URL=your_supabase_url
   SUPABASE_SERVICE_KEY=your_supabase_service_key
   GEMINI_API_KEY=your_gemini_api_key
   ANTHROPIC_API_KEY=your_anthropic_api_key
  1. 1.Installation
bash
   # Install dependencies
   pip install -r requirements.txt

Running Locally

bash
python main.py

Deployment on Hugging Face Spaces

  1. 1.Create a new Space on Hugging Face
  2. 2.Choose Docker as the Space type
  3. 3.Add your environment variables as Secrets in the Space settings
  4. 4.Push your code to the Space repository

The Docker container will automatically build and start running the application.

Architecture

The system follows an event-driven architecture:

  1. 1.Images are uploaded to Supabase Storage
  2. 2.A record is inserted into the jobs table
  3. 3.The backend receives real-time notifications via WebSocket
  4. 4.Images are processed in batches with a 7-second window
  5. 5.OCR and AI analysis are performed concurrently
  6. 6.Results are stored back in the database

Database Schema

The system uses two main tables:

  • —batches: Manages groups of image processing jobs
  • —jobs: Tracks individual image processing tasks

For detailed schema information, refer to db_setup.sql.

Contributing

  1. 1.Fork the repository
  2. 2.Create a feature branch
  3. 3.Commit your changes
  4. 4.Push to the branch
  5. 5.Create a Pull Request

License

MIT