Backblaze Labs
All projects
SampleSample · unsupportedUnsupported · no SLA

DataComp Image-Text Filtering

DataComp-style image-text dataset filtering on Backblaze B2: stream WebDataset shards from object storage, score image-text alignment with CLIP, and write filtered shards plus quality metrics back to B2 — no local staging, no database. Full-stack Next.js + FastAPI sample for VLM-pretraining data curation.

Built for: ML engineers and data curators building vision-language-model pretraining datasets, and teams evaluating Backblaze B2 as the storage layer for large-scale data curation.

Get the codeCheck project fit

MIT licensed · Updated Aug 18, 2026

Dashboard with run totals, pairs-kept, and average-reduction stats above a recent-runs table
Filter-run totals, pairs kept, and average storage reduction, over a table of the most recent runs.

What it's built with

Each piece of the stack, and the job it does in this project.

Next.js 16 + React 19
Frontend — App Router, Tailwind v4, shadcn/ui
FastAPI
Python 3.12 backend with a strict layered architecture
open_clip (CLIP ViT-B-32)
On-device image-text alignment scoring engine
WebDataset
.tar shard format streamed from and written back to B2
Backblaze B2
Sole storage layer via the S3-compatible API

Core capabilities

What the project does out of the box, before you write any code of your own.

  • DataComp-style CLIP-score filtering

    Stream shards from B2, score image-text alignment with CLIP, and keep the top-percentile pairs (DataComp's clip_score baseline).

  • Configurable baseline filters

    CLIP-score percentile, min image resolution, caption-length bounds, and near-duplicate removal — the clip_score / basic / image_based / text_based families, chosen on the Run form.

  • Filter Runs, full lifecycle in the UI

    Create, read, edit, delete, and run filter jobs. B2 manifests are the sole store — no database.

  • Pool Explorer

    Open a WebDataset shard to inspect the image-text pairs inside it: thumbnail, caption, CLIP score, and kept/dropped.

  • Bucket Explorer

    Browse every object in the bucket (pool, filtered, manifests, metrics) with preview, download, and delete.

A closer look

More screens from the running project.

Filter Runs list showing each run's strategy, status, kept-vs-in counts, and reduction
Every CLIP filter job with its strategy, status, kept/in counts, and reduction, plus the full create/run/delete lifecycle.
Run detail with configuration, results, per-shard metrics, and per-pair CLIP-score decisions
A completed run's configuration and results alongside per-shard metrics and the per-pair CLIP scores with kept/dropped decisions.
Pool Explorer showing a grid of image-text pairs from a raw WebDataset shard
Open a WebDataset shard to inspect the image-text pairs inside it: thumbnail, caption, and resolution.
Ingest page with a drag-and-drop zone for direct-to-B2 uploads
Upload raw WebDataset shards or image-text assets straight into B2 with a presigned, direct-to-bucket PUT.

Choose it for the right job

DataComp Image-Text Filtering is a head start for a specific shape of project, not a supported, general-purpose product.

Use it when

  • You want a working reference for a DataComp-style CLIP-score filtering pipeline that streams WebDataset shards from object storage.
  • You want to run real CLIP scoring over an image-text pool and inspect which pairs were kept vs dropped and why.
  • You're evaluating Backblaze B2 as the sole storage layer for vision-language-model pretraining data curation — no database.

Choose another path when

  • You need petabyte-scale throughput as-is — the demo holds each shard in memory; production would stream via WebDataset's S3 reader and run the engine on a GPU fleet.
  • You need multi-tenant access control — the UI is unauthenticated and operates bucket-wide (single-tenant demo).
Sample · unsupported

Project status and support

Community sample with no SLA — file issues on the GitHub repository, but support is best-effort only.