Schema and Data Migrations: Why They Should Be Separate

Rails developers often mix schema and data work in the same timestamped migration file. The framework handles structural changes well—adding columns, creating tables, modifying indexes. Long-running data transforms in that same pipeline cause deployment, rollback, and bootstrap problems that tend to surface months later.

Vesa Vänskä's 2014 essay on ActiveRecord migrations best practices states plainly: "Migrations should not contain seed data." Yet teams continue to bundle data transformations into schema migrations, creating problems that surface during deployments, rollbacks, and database bootstrapping.

Schema vs data migrations

Schema migrations and data migrations appear similar—both modify the database, both use timestamped files, both run during deployment. This superficial similarity masks fundamental differences in purpose, execution requirements, and failure modes.

Schema migrations change structure. They should be fast, automatic, and reversible. Data migrations transform content. They may take hours, require operator oversight, and often cannot be reversed. Mixing them creates a category error that manifests as deployment failures, incomplete databases, and impossible rollbacks.

Bootstrap and schema:load

When you run db:schema:load to bootstrap a new database, Rails loads only the schema structure. Data migrations bundled into schema migrations never execute. Your database structure exists, but the data remains in its previous state. This breaks applications that assume data transformations have occurred.

Teams discover this when staging environments behave differently than production, or when new developers struggle to set up local databases. The workaround—running all migrations sequentially—defeats the purpose of schema.rb as a snapshot of the current database state.

Deployment automation

Most CI/CD pipelines automatically run db:migrate during deployment. This works for schema changes, which are fast and predictable. Data migrations, however, may require:

• Operator approval before execution
• Specific timing windows (e.g., during low-traffic periods)
• Manual verification of results
• Rollback plans if execution fails

Automating data migrations removes human judgment from critical data transformations. A migration that updates millions of records should not run automatically at 2 AM during a deployment. Yet teams continue to mix data transformations into schema migrations, creating this exact scenario.

Large updates and transactions

Some developers wrap data migrations in transactions, assuming this provides safety. For large data transformations, transactions become memory-intensive and can cause database unresponsiveness. Wrapping a migration that updates ten million records in a transaction may exhaust available memory or lock tables for hours.

Data migrations require batching, progress tracking, and the ability to pause and resume. These requirements conflict with the transaction model that works well for schema changes.

Testing data migrations

Rails doesn't test schema migrations by default—they're considered framework-level operations. Data migrations inherit this pattern, but teams often want to test critical data transformations. Mixing schema and data migrations makes it unclear what requires testing and what doesn't.

The data-migration gem provides in-file RSpec tests as an option, embedded directly in migration files. This acknowledges that data migrations may need testing while maintaining the Rails convention that migrations themselves aren't part of the main test suite.

Why keep them separate

Schema and data migrations differ in purpose, runtime, and rollback expectations:

• Schema migrations: Fast, automatic, reversible, structural changes
• Data migrations: Potentially slow, operator-controlled, often irreversible, content transformations

The data-migration gem provides a framework that mimics ActiveRecord migrations but is designed for data transformations. It uses plain Ruby classes with a perform method, wraps them in ActiveJob for background execution, and tracks them via ActiveRecord models for auditing. This design follows familiar patterns while accommodating the unique requirements of data transformations.

Data migrations should be planned beforehand, scheduled as release events, and executed with operator oversight. They may need batching, pause/resume capabilities, and progress tracking. These features don't belong in schema migrations, which must remain fast and automatic.

The Practical Alternative

Instead of mixing schema and data, separate them:

Create a schema migration to add the column. Then create a separate data migration to populate it. Run the schema migration automatically during deployment. Run the data migration manually when appropriate, with operator oversight and monitoring.

This separation provides control, visibility, and safety. Schema changes remain fast and automatic. Data transformations remain controlled and observable. Each operation uses the tooling appropriate to its requirements.

Conclusion

The temptation to mix schema and data migrations stems from convenience—one file, one execution, one commit. But this convenience creates operational debt that compounds over time. Deployment failures, incomplete databases, and impossible rollbacks are the inevitable results.

Separating schema and data migrations requires discipline, but the benefits are immediate: better control over when data transformations execute, proper handling of database bootstrapping, and the ability to test and monitor data migrations independently. The data-migration gem provides the framework needed to implement this separation while maintaining familiar Rails patterns.

As Vänskä noted over a decade ago, migrations should not contain seed data. The same principle applies to all data transformations. Structure and content are different concerns, and they deserve different tooling.