Skip to content

Latest commit

 

History

History
258 lines (197 loc) · 6.84 KB

API_README.md

File metadata and controls

258 lines (197 loc) · 6.84 KB

ML-Sharp SBS Video Conversion API

A FastAPI-based REST API service for converting 2D videos to 3D Side-by-Side (SBS) format using Gaussian Splatting.

Features

  • Async Job Queue: Process multiple videos with background job management
  • Progress Tracking: Monitor frame-by-frame progress of video conversion
  • Dynamic Resolution: Uses input video dimensions for render output
  • Flexible Parameters: Configure precision, batch size, frame skip, and more
  • Auto-Documentation: Built-in OpenAPI/Swagger documentation

Installation

# Install API dependencies
pip install -r requirements-api.txt

# Or install directly
pip install fastapi uvicorn python-multipart

Quick Start

# Start the API server (default: http://0.0.0.0:8000)
python api_service.py

# With custom settings
python api_service.py --host 0.0.0.0 --port 8080 --preload --workers 2

CLI Options

Option Default Description
--host 0.0.0.0 Host to bind to
--port 8000 Port to bind to
--preload false Preload ML model on startup
--workers 1 Max concurrent processing jobs
--log-level INFO Logging level

API Documentation

Once running, visit:

  • Swagger UI: http://localhost:8000/docs
  • ReDoc: http://localhost:8000/redoc

API Endpoints

POST /jobs/create - Create Conversion Job

Submit a video for SBS conversion.

Request: multipart/form-data

Field Type Default Description
video File Required Video file (.mp4, .mov, .avi, .mkv, .webm)
precision string "float32" Precision: float32 or float16
batch_size int 1 Batch size (1-8, higher = faster but more GPU memory)
frame_skip int 1 Process every Nth frame (1 = all frames)
opacity_threshold float 0.0 Halo removal threshold (0.0-1.0)
stereo_offset float 0.015 Stereo camera offset (0.0-0.1)
brightness_factor float 1.0 Brightness adjustment (0.1-3.0)

Response:

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "queued",
  "message": "Job created successfully. Check /jobs/{job_id} for status."
}

GET /jobs/{job_id} - Get Job Status

Check the progress of a job.

Response:

{
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "running",
  "stage": "processing_frames",
  "progress": 0.45,
  "processed_frames": 450,
  "total_frames": 1000,
  "current_frame": 450,
  "created_at": "2026-07-28T12:00:00",
  "started_at": "2026-07-28T12:00:05",
  "completed_at": null,
  "error_message": null,
  "output_files": [],
  "input_filename": "video.mp4",
  "settings": {...}
}

Status values: queued, running, completed, failed, cancelled

Stage values: queued, downloading_model, extracting_audio, processing_frames, encoding_video, completed, failed

GET /jobs - List All Jobs

List all jobs with optional filtering.

Query Params:

  • status (optional): Filter by status (queued, running, completed, failed)
  • limit (optional): Max results (default: 100)

DELETE /jobs/{job_id} - Cancel Job

Cancel a running or queued job.

GET /download/{filename} - Download Output

Download a generated SBS video file.

DELETE /jobs/{job_id}/output - Delete Output Files

Delete output files for a completed job.

GET /status - Server Status

Get server status and statistics.

Response:

{
  "status": "ok",
  "device": "cuda",
  "model_loaded": true,
  "cuda_available": true,
  "active_jobs": 1,
  "total_jobs_processed": 42
}

Usage Examples

cURL

# Create a job
curl -X POST "http://localhost:8000/jobs/create" \
  -F "video=@myvideo.mp4" \
  -F "precision=float16" \
  -F "batch_size=2" \
  -F "frame_skip=1"

# Check status
curl "http://localhost:8000/jobs/{job_id}"

# Download result
curl "http://localhost:8000/download/001_myvideo_SBS.mp4" -o output.mp4

Python

import requests

# Create job
with open("myvideo.mp4", "rb") as f:
    response = requests.post(
        "http://localhost:8000/jobs/create",
        files={"video": f},
        data={
            "precision": "float16",
            "batch_size": 2,
            "frame_skip": 1
        }
    )
job_id = response.json()["job_id"]

# Poll for completion
while True:
    status = requests.get(f"http://localhost:8000/jobs/{job_id}").json()
    print(f"Progress: {status['progress']:.1%} - Frame {status['processed_frames']}/{status['total_frames']}")
    if status["status"] in ("completed", "failed"):
        break
    time.sleep(5)

# Download result
if status["status"] == "completed":
    filename = status["output_files"][0]
    video = requests.get(f"http://localhost:8000/download/{filename}")
    with open(filename, "wb") as f:
        f.write(video.content)

JavaScript

// Create job
const formData = new FormData();
formData.append("video", fileInput.files[0]);
formData.append("precision", "float16");
formData.append("batch_size", "2");

const response = await fetch("http://localhost:8000/jobs/create", {
  method: "POST",
  body: formData
});
const { job_id } = await response.json();

// Poll for completion
const pollStatus = async () => {
  const status = await fetch(`http://localhost:8000/jobs/${job_id}`).then(r => r.json());
  console.log(`Progress: ${(status.progress * 100).toFixed(1)}%`);
  if (status.status !== "running" && status.status !== "queued") {
    return status;
  }
  await new Promise(r => setTimeout(r, 2000));
  return pollStatus();
};

const finalStatus = await pollStatus();

How It Works

  1. Upload: Video is saved to temporary storage
  2. Queue: Job enters the processing queue
  3. Extract Audio: Audio track is extracted using ffmpeg
  4. Process Frames:
    • Each frame is converted to 3D Gaussian Splat
    • Stereo pairs are rendered (left/right eye)
    • Frames are saved as PNG sequence
  5. Encode Video: Frames are encoded back to MP4 with audio
  6. Output: SBS video is ready for download

Output Format

  • Resolution: INPUT_WIDTH * 2 x INPUT_HEIGHT (side-by-side)
  • Video Codec: H.264 (libx264)
  • Pixel Format: YUV420P
  • Quality: CRF 18 (high quality)

Hardware Requirements

  • GPU: NVIDIA CUDA GPU required for SBS rendering
  • Memory: Minimum 8GB GPU RAM (more for higher resolutions/batch sizes)
  • CPU: Any modern CPU for audio extraction and video encoding

Performance Tips

Parameter Effect
precision=float16 2x faster inference, slightly less quality
batch_size=2-4 Faster processing, needs more GPU memory
frame_skip=2 Process every 2nd frame, 2x faster, lower FPS output
opacity_threshold=0.1 Removes faint splats, reduces "halo" artifacts

License

See LICENSE file in the project root.