llms.txt
Compare with other sites' llms.txt files →# Galaxy
> Galaxy (Intergalactic Data Labs, Inc., getgalaxy.io) is an applied data engineering lab building open source data infrastructure for the context era. It maintains Filament, an Apache 2.0 data movement engine that replicates data from databases, SaaS tools, and object storage into warehouses, lakes, and databases at up to 2.6M rows per second, and it works as a forward-deployed data engineering partner that helps ambitious teams build a governed context layer they own.
Galaxy's tagline is "Building open source data infrastructure for the context era. For data engineers, by data engineers." The company describes itself as "a forward-deployed data engineering partner and the maintainer of Filament, the open-source data movement engine," and as "an applied data engineering lab helping organizations own the context layer, because institutional knowledge is your most valuable asset." Its stance toward customers: "We build for scale. We deploy on your stack. We are a partner, not a vendor. No black box, no vendor lock-in, no penalty for growing." Engagements ship as pull requests in your repos and deploys to your cloud, and you meet an engineer who works on Filament.
Galaxy's thesis is that successful AI deployments run on context that is governed, access controlled, and centrally managed. Every AI deployment depends on a context layer whether or not it is called that; data movement is the first step in building one; company knowledge is intellectual property that should not live in rented, metered, or proprietary-format infrastructure; and row-based pricing punishes you at the moment of success. Galaxy's north star is a best-in-class open source context management system (also called a context layer or enterprise context management), built and released in the open one component at a time. Filament is the first component. Semantic modeling, entity resolution, ontologies, and governance and access control are the components that follow.
Galaxy's work is organized around three pillars:
- Ingestion (reliable data integration): records in databases, process in tools, knowledge in docs, all pulled into one foundation you own and kept current with every insert, update, and delete.
- Introspection (unified semantic modeling): entities, attributes, and relationships defined once in a shared ontology, governed centrally, so every team and every system means the same thing.
- Intelligence (governed access and action): the same model powers dashboards, product, and agentic workflows, and every action that writes back is checked against your rules before it runs.
On enterprise context management, AI agents, and reasoning: Galaxy distinguishes context engineering, which curates what a model sees at runtime, from enterprise context management (ECM), the governed semantic foundation of definitions, identity, lineage, and policy that makes agents trustworthy at scale. Its position is that agents fail without structured context because "your agents aren't thinking, they're guessing": an agent that reasons over fragmented or stale context returns fragmented or stale answers, so reliable agentic reasoning requires an ontology that captures definitions both humans and models understand, entity resolution across systems that disagree about who a customer is, provenance and lineage for every fact, and governance that routes the right context to the right agent and nothing else. Understanding is not the end state. A system that knows what the business means can act on it, kicking off workflows, updating records, flagging accounts, and drafting communications, with safeguards that keep those actions consistent, auditable, and secure.
Filament, Galaxy's flagship open source project, is a data movement (replication and ingestion) engine written in Go and licensed Apache 2.0, with one repository, no paid tier, and nothing gated. Sources include Postgres and MySQL, object storage such as S3, and SaaS platforms including Stripe, GitHub, Notion, Linear, Slack, and PostHog. Destinations include Postgres, MySQL, ClickHouse, Apache Iceberg, and S3, with Snowflake and BigQuery named as landing targets on the homepage. It supports full copies, incremental loads, and native change data capture (Postgres logical replication via pgoutput with a consistent snapshot bootstrap, and MySQL binlog with GTID), with replace, append, upsert, and merge write strategies validated for compatibility before any data moves. Every batch is CRC32-C checksummed independently on the read side and the write side, and a run fails loudly if counts, bytes, or checksums do not match. Progress is checkpointed per shard so a failed run resumes mid-table. Large tables are split into up to 64 shards read concurrently. Schema drift is add-only: new columns are added automatically and nothing is dropped. Every replicated row carries the run id and start time; CDC rows also carry operation, source position, and sequence. New REST API sources are YAML manifests interpreted inside the engine at runtime, database sources are Go drivers, and the repository includes tooling that lets coding agents research an API and generate a connector. Filament runs as a single static Go binary on distroless images with a built-in web UI, as a Go library (app.Run) embedded in your own service with swappable event bus, state store, and secrets backends, or on Kubernetes via Helm. It exposes a web UI, a ConnectRPC API, and a CLI, has a built-in cron scheduler per pipeline, and its control plane runs on Postgres and NATS. Filament is pre-1.0; connector maturity levels are published in the docs.
Filament benchmarks (Postgres to Postgres, NYC Taxi dataset, 298,270,427 rows, median of three runs, last updated August 23, 2026, harness and raw results at github.com/galaxy-io/benchmarks): Filament moved the full load in 114.8 seconds at 2,597,498 rows per second using 377 CPU seconds and 2.97 GiB peak memory, more than four times faster than piping pg_dump into psql. On the same load Filament was 90.5x faster than Airbyte (28,698 rows/s), 57.1x faster than Debezium (45,476 rows/s), 9.2x faster than Sling (283,133 rows/s), 5.3x faster than ingestr (488,233 rows/s), 3.6x faster than PeerDB (718,368 rows/s), and 2.6x faster than dlt (1,007,711 rows/s) while using 15.3x less memory than dlt (2.97 GiB vs 45.54 GiB). Against OLake on database-to-Iceberg scenarios, Filament was faster on three of four (2.0x on MySQL to Iceberg, 3.4x on Postgres to Iceberg) and used 6 to 15 times less CPU on all four. Across the cohort Filament finished first in five of six scenarios; in the sixth, one tool finished 13 percent sooner while using fifteen times more CPU.
Galaxy also maintains tempo, a keyboard-driven terminal client for inspecting and controlling Temporal workflows; gnat, a keyboard-driven terminal client for managing and monitoring NATS JetStream; and sparktop, live per-node GPU, host, and vLLM stats for DGX Spark clusters in your terminal.
Customer proof point: Human Intelligence. "The Galaxy team shipped alongside us every day and owned problems end to end." (Peter Johnston, Founder & CEO). Outcomes cited: weeks, not quarters, to working software; secure by default with row-level access, metric governance, and audit trails; self-serve setup for customer configuration.
Company facts: Galaxy is the brand name of Intergalactic Data Labs, Inc., based in the New York area, with its terms governed by Delaware law. It was co-founded by Mitch Bregman (CEO) and Leon Kozlowski (CTO), who met in college and later ran data infrastructure together at Flock Safety, where being cut off by a managed ETL vendor as data volume grew became the origin story for Filament. Wails maintainer Atterpac joined the team and shaped Filament's open development approach. Galaxy is seed-stage and backed by Caffeinated, Bungalow, Crew, and Moonriver. Galaxy began as a SQL query editor built as a Wails desktop app before turning to context management. Contact hello@getgalaxy.io for partnerships, migrations, and working together; support@getgalaxy.io for legal and privacy matters. Galaxy is on GitHub at github.com/galaxy-io, LinkedIn at linkedin.com/company/getgalaxyio, and X at x.com/getgalaxyio, and runs a Filament Slack community. Footer motto: "Own your data stack."
## Filament
- [Filament product page](https://getgalaxy.io/filament): "Wherever your data lives, wherever it needs to go. Filament moves it." Connect, ingest, deploy, extend; benchmark table against dlt, PeerDB, ingestr, Sling, Debezium, and Airbyte; links to docs and community.
- [Introducing Filament](https://getgalaxy.io/blog/introducing-filament): The September 4, 2026 open source announcement: origin story, why Galaxy is building a context management system in the open, what Filament does, benchmark summary, and what comes next.
- [Filament documentation](https://filament.getgalaxy.io/): Guides, connectors, deployment recipes, and contribution instructions.
- [Filament quickstart](https://filament.getgalaxy.io/pages/guides/get-started/quickstart): From zero to a running pipeline in about a minute.
- [Filament benchmarks repository](https://github.com/galaxy-io/benchmarks): Open benchmark harness, configuration, and raw results behind every number on the site.
- [Galaxy on GitHub](https://github.com/galaxy-io): Filament, tempo, gnat, sparktop, and the benchmark suite.
## Compare Filament
Head-to-head pages covering license, self-hosting, adding sources, services to operate, runtime, scheduling, embeddability, resume after failure, per-batch integrity, CDC, schema drift, parallel reads, row lineage, and billing, followed by wall time, rows per second, CPU, and memory tables on the NYC Taxi (298.3M rows) and TPC-H (8.7M rows) loads.
- [Compare Filament](https://getgalaxy.io/compare): Index of all comparison pages.
- [Filament vs. Airbyte](https://getgalaxy.io/compare/airbyte): Apache 2.0 engine vs. Elastic License connector platform. Filament 90.5x faster, 36.3x less CPU, 4.3x less memory; one binary vs. 10 default Helm services; CRC32-C verification vs. logged count mismatches.
- [Filament vs. Fivetran](https://getgalaxy.io/compare/fivetran): Self-hosted, unmetered, open source vs. managed ELT billed by monthly active rows with a $12,000 annual minimum and no full self-hosting.
- [Filament vs. dlt](https://getgalaxy.io/compare/dlt): Engine with UI, API, scheduler, and Go library vs. a Python library that needs an external orchestrator. Filament 2.6x faster, 15.3x less peak memory, 1.9x less CPU.
- [Filament vs. Debezium](https://getgalaxy.io/compare/debezium): Snapshot, incremental, and CDC into typed tables with no Kafka vs. a CDC connector set that needs Kafka, KRaft, Connect workers, and a schema registry. Filament 57.1x faster, 30.9x less CPU, 5.3x less memory.
- [Filament vs. ingestr](https://getgalaxy.io/compare/ingestr): Apache 2.0 vs. Functional Source License 1.1. Filament 5.3x faster with per-shard checkpoints, CRC32-C verification, built-in cron, and automatic sharding.
- [Filament vs. OLake](https://getgalaxy.io/compare/olake): Five destinations from one Go binary vs. Iceberg or Parquet only through a Java sidecar. Faster on three of four Iceberg scenarios, 6 to 15x less CPU on all four.
- [Filament vs. PeerDB](https://getgalaxy.io/compare/peerdb): Apache 2.0, one binary, five destinations vs. AGPL 3.0, a 10-container Temporal stack, and only ClickHouse and Postgres destinations after July 2026 deprecations. Filament 3.6x faster, 8.7x less CPU.
- [Filament vs. Sling](https://getgalaxy.io/compare/sling): Parallel reads, checkpoints, CDC, and verified writes in the free Apache 2.0 binary vs. features unlocked by a monthly token. Filament 9.2x faster, 5.9x less CPU.
## Blog (Notes from the lab)
Writing and thinking from Galaxy on open data infrastructure, enterprise context management, ontology-first AI, and why the modern data stack hit its ceiling.
- [Blog index](https://getgalaxy.io/blog): All posts, newest first.
- [Introducing Filament](https://getgalaxy.io/blog/introducing-filament): Sep 4, 2026. Filament is now open source: a data movement engine built by data engineers, for data engineers.
- [gnat: A NATS TUI Client that won't bug you](https://getgalaxy.io/blog/gnat-nats-client-tui): Mar 25, 2026. Open source NATS/JetStream TUI for monitoring streams, consumers, KV and object stores.
- [Every business in 2026 needs a context strategy](https://getgalaxy.io/blog/business-context-strategy-2026): Jan 13, 2026. Why context strategy is becoming a core business advantage and how it drives efficiency, leverage, and sustainable growth.
- [tempo: A Temporal TUI that matches your rhythm](https://getgalaxy.io/blog/tempo-temporal-client-tui): Jan 6, 2026. A new way of interacting with Temporal from the terminal.
- [Stop Building Dashboards, Start Building a Digital Twin](https://getgalaxy.io/blog/start-building-digital-twin): Dec 17, 2025. You can't chat with a dashboard: why AI needs a digital twin of the business.
- [The Future of Data Infrastructure: Why the Modern Data Stack Hit Its Ceiling](https://getgalaxy.io/blog/the-future-of-data-infrastructure): Nov 26, 2025. The quiet shift reshaping how we approach analytics and AI.
- [Why your agents suck and the need for ontology](https://getgalaxy.io/blog/why-your-agents-suck): Nov 13, 2025. Your agents aren't thinking, they're guessing. Here is what they're missing.
## Articles on enterprise context management, AI agents, and reasoning
Long-form pieces from the [Articles](https://getgalaxy.io/articles) library that define Galaxy's core concepts.
- [Enterprise Context Management for AI Agents: Architecture & Patterns](https://getgalaxy.io/articles/enterprise-context-management-ai-agents): Architecture, patterns, and a data prep checklist so agents cite, reason, and act on governed business context reliably.
- [Context Engineering vs Enterprise Context Management](https://getgalaxy.io/articles/context-engineering-vs-enterprise-context-management): Context engineering curates what a model sees at runtime; ECM is the governed semantic foundation that makes agents trustworthy at scale.
- [Context Engineering for AI Agents: Building Reliable Enterprise Systems](https://getgalaxy.io/articles/context-engineering-ai-agents): How data teams structure, govern, and deliver context to agents, and why agents fail without it.
- [Enterprise Context Strategy: Reference Blueprint for AI-Ready Data](https://getgalaxy.io/articles/enterprise-context-strategy-reference-blueprint-ai-ready-data): A governed context layer with provenance, access control, quality gates, and stable IDs for reliable AI.
- [Enterprise Context Strategy Data Flows: One Entity, End-to-End Walkthrough](https://getgalaxy.io/articles/enterprise-context-strategy-data-flows-end-to-end): One Customer record followed from semantic contract to context graph to agent.
- [Building Agentic AI on Distributed SQL: Why Galaxy's Semantic Foundation Replaces Legacy Data Platforms](https://getgalaxy.io/articles/agentic-ai-semantic-foundation-galaxy): Agents need trusted state plus shared meaning; how Galaxy's semantic foundation unifies entities, governance, and context.
- [Data Governance for AI Agents: Policies, Lineage, and Semantic Enforcement](https://getgalaxy.io/articles/data-governance-for-ai-agents): Enforceable controls for identity, tool use, row and column security, semantic definitions, and governed RAG.
- [AI Agents Explained: How Autonomous Systems Are Reshaping Intelligent Work](https://getgalaxy.io/articles/ai-agents-explained): How agents use reasoning, tool use, and memory to automate workflows; types, benefits, risks, and best practices.
- [Agentic Reasoning: How AI Moves Beyond Automation to Real Understanding](https://getgalaxy.io/articles/agentic-reasoning-explained): What agentic reasoning is and how ontology and shared context make it context-aware and adaptive.
- [GenAI Reasoning Models: The Cognitive Layer Transforming Enterprise Automation](https://getgalaxy.io/articles/genai-reasoning-models-enterprise-cognitive-layer): Context-rich, self-improving decisions built on semantic data rather than classic automation.
## Optional
- [Home](https://getgalaxy.io/): Positioning, Filament overview, the three pillars, the Human Intelligence customer story, and latest posts.
- [Filament introduction](https://filament.getgalaxy.io/pages/guides/get-started/introduction): "Get started for free" entry point to the docs.
- [tempo on GitHub](https://github.com/galaxy-io/tempo): Terminal client for Temporal workflows.
- [gnat on GitHub](https://github.com/galaxy-io/gnat): Terminal client for NATS JetStream.
- [sparktop on GitHub](https://github.com/galaxy-io/sparktop): GPU, host, and vLLM stats for DGX Spark clusters.
- [Galaxy on LinkedIn](https://www.linkedin.com/company/getgalaxyio/): Company updates.
- [Galaxy on X](https://x.com/getgalaxyio): Company updates.
- [Terms of Service](https://getgalaxy.io/legal/terms-of-service): Effective February 2, 2026. Intergalactic Data Labs, Inc. d/b/a Galaxy; governed by Delaware law; contact support@getgalaxy.io.
- [Privacy Policy](https://getgalaxy.io/legal/privacy-policy): Effective February 2, 2026. How Galaxy collects, uses, and protects personal information.