Documentation
Learn about SkyLaneAI's multi-agent AI system and how it protects aerial vehicles from sky hazards.
Overview
SkyLaneAI is a safety-critical system designed to protect flying taxis and other aerial vehicles from sky hazards. Using advanced AI technology, it detects birds, drones, balloons, and kites in video feeds and provides intelligent, context-aware alerts.
The system employs a sophisticated multi-agent AI architecture powered by LangGraph and OpenAI, where specialized agents work together to analyze detections, assess threats, and generate actionable recommendations for pilots.
Key Features
Advanced zero-shot object detection using YOLO-World, capable of identifying custom classes including birds, drones, balloons, and kites with high accuracy and minimal latency.
Four specialized agents work together: Context Agent analyzes patterns, Action Agent recommends pilot actions, Message Agent crafts alerts, and Priority Agent ranks threats by urgency.
AI-generated natural language alerts that include threat assessment, detection details, and actionable pilot recommendations based on aviation safety protocols.
Upload pre-recorded videos for detailed analysis with interactive timeline, detection filtering, and comprehensive alert review capabilities.
Multi-Agent System Architecture
1. Context Agent (Rule-based)
Analyzes detection data to enrich context with size estimation, screen position, and initial threat level calculation. Fast and deterministic with no API costs.
- Calculates bounding box area and size category (small, medium, large)
- Determines screen position (upper-left, center, lower-right, etc.)
- Computes threat level (low, moderate, high, critical)
2. Action Agent (LLM-powered)
Uses OpenAI GPT to generate actionable pilot recommendations following aviation safety protocols.
- PRIMARY ACTION: Most critical immediate action
- SECONDARY ACTION: Follow-up or alternative action
- REASONING: Brief explanation of recommendations
- URGENCY: Advisory, caution, urgent, or immediate
3. Message Agent (LLM-powered)
Generates natural language alert messages with professional, aviation-focused tone.
- Creates structured alerts with title and emoji
- Includes Detection Details and Threat Assessment sections
- Provides scannable, actionable information for pilots
- Fallback to rule-based messages if LLM fails
4. Priority Agent (Rule-based)
Scores and ranks alerts using a weighted system to help pilots focus on critical threats first.
- Threat level (40% weight)
- Action urgency (30% weight)
- Detection confidence (20% weight)
- Object size (10% weight)
How It Works
1. Video Upload
Upload a video file through the web interface. Supported formats include MP4, AVI, MOV, and MKV.
2. Object Detection
Each video frame is analyzed using YOLO-World zero-shot detection to identify potential sky hazards including birds, drones, balloons, and kites using custom-defined classes.
3. Multi-Agent Analysis
Detected hazards flow through the multi-agent pipeline: Context Agent enriches data, Action Agent recommends pilot actions, Message Agent crafts alerts, and Priority Agent ranks threats by urgency.
4. Interactive Results
View detections with bounding box overlays on the video player, explore the interactive timeline, filter by hazard type, and review AI-generated alerts with actionable recommendations.
Threat Levels (Context Agent)
Large hazard with high confidence, significant screen coverage. Requires immediate action.
Hazard detected with good confidence and considerable size. Urgent monitoring and potential evasive action.
Hazard present with reasonable confidence. Continue monitoring and be prepared for course adjustment.
Small or distant hazard. Maintain awareness but no immediate action required.
Technology Stack
Frontend
- • Next.js with App Router
- • React with TypeScript
- • Tailwind CSS
- • shadcn/ui components
Backend
- • FastAPI (Python)
- • YOLO-World zero-shot detection
- • OpenCV for video processing
- • File-based storage (temporary)
AI Multi-Agent System
- • LangGraph for agent orchestration
- • OpenAI GPT (gpt-4o-mini)
- • 4 specialized agents (2 LLM, 2 rule-based)
- • Fallback logic for reliability
Coming Soon
- • Live camera streaming (WebRTC)
- • Time-to-Contact (TTC) calculation
- • Persistent database storage
- • Multi-camera support