Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Now that we have our Movie database schema ready we can proceed to create, read, update and delete records.
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
To get started with CQL in your Crystal project, follow these steps:
First, add CQL to your project by including it in the shard.yml
file:
dependencies:
cql:
github: azutoolkit/cql
version: "~> 0.1.0"
Run the following command to install the dependencies:
shards install
Set up your database connection by specifying the adapter and connection URL. This is done by configuring the database in your code, as follows:
"postgres://user:password@localhost:5432/database_name"
In this example, we’re using PostgreSQL. You can change the URL according to your database (MySQL, SQLite, etc.).
Now you can define your schema and run migrations (explained in later sections).
CQL is a powerful library designed to simplify and enhance the management and execution of SQL queries in the Crystal programming language. It provides utilities for building, validating, and executing SQL statements, ensuring better performance and code maintainability.
Query Builder: Programmatically create complex SQL queries.
Insert, Update, Delete Operations: Perform CRUD operations with ease.
Repository Pattern: Manage your data more effectively using CQL::Repository(T)
.
Active Record Pattern: Work with your data models using CQL::Record(T)
.
Add this to your application's shard.yml
:
dependencies:
cql:
github: azutoolkit/cql
Then, run the following command to install the dependencies:
shards install
Define the schema for your database tables:
AcmeDB2 = CQL::Schema.build(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]) do
table :movies do
primary :id, Int64, auto_increment: true
text :title
end
table :screenplays do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :content
end
table :actors do
primary :id, Int64, auto_increment: true
text :name
end
table :directors do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :name
end
table :movies_actors do
primary :id, Int64, auto_increment: true
bigint :movie_id
bigint :actor_id
end
end
With the schema in place, you can start executing queries:
q = AcmeDB.query
user = q.from(:users).where(id: 1).first!(as: User)
puts user.name
Insert new records into the database:
q = CQL::Query.new(schema)
q.insert
.into(:users)
.values(name: "Jane Doe", email: "jane@example.com")
.last_insert_id
Update existing records:
u = AcmeDB.update
u.table(:users)
.set(name: "Jane Smith")
.where(id: 1)
.commit
Delete records from the database:
d = AcmeDB.delete
d.from(:users).where(id: 1).commit
Utilize the repository pattern for organized data management:
user_repository = CQL::Repository(User, Int64).new(schema, :users)
# Create a new user
user_repository.create(name: "Jane Doe", email: "jane@example.com")
# Fetch all users
users = user_repository.all
users.each { |user| puts user.name }
# Find a user by ID
user = user_repository.find!(1)
puts user.name
# Update a user by ID
user_repository.update(1, name: "Jane Smith")
Work with your data using the Active Record pattern:
struct Actor < CQL::Record(Int64)
db_context AcmeDB2, :actors
getter id : Int64?
getter name : String
def initialize(@name : String)
end
end
struct Movie < CQL::Record(Int64)
db_context AcmeDB2, :movies
has_one :screenplay, Screenplay
many_to_many :actors, Actor, join_through: :movies_actors
has_many :directors, Director, foreign_key: :movie_id
getter id : Int64?
getter title : String
def initialize(@title : String)
end
end
struct Director < CQL::Record(Int64)
db_context AcmeDB2, :directors
getter id : Int64?
getter name : String
belongs_to :movie, foreign_key: :movie_id
def initialize(@name : String)
end
end
struct Screenplay < CQL::Record(Int64)
db_context AcmeDB2, :screenplays
belongs_to :movie, foreign_key: :movie_id
getter id : Int64?
getter content : String
def initialize(@movie_id : Int64, @content : String)
end
end
struct MoviesActors < CQL::Record(Int64)
db_context AcmeDB2, :movies_actors
getter id : Int64?
getter movie_id : Int64
getter actor_id : Int64
has_many :actors, Actor, :actor_id
def initialize(@movie_id : Int64, @actor_id : Int64)
end
end
Once you've defined your database schema using CQL's Schema.define
, the next step is to initialize the database by creating the necessary tables and structures. In this guide, we will walk through how to define your schema and use Schema.init
to initialize the database.
Here's an example where we define a schema for a movie database using CQL. The schema includes tables for movies
, screenplays
, actors
, directors
, and a join table movies_actors
to link movies and actors.
AcmeDB2 = CQL::Schema.define(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :movies do
primary :id, Int64, auto_increment: true
text :title
end
table :screenplays do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :content
end
table :actors do
primary :id, Int64, auto_increment: true
text :name
end
table :directors do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :name
end
table :movies_actors do
primary :id, Int64, auto_increment: true
bigint :movie_id
bigint :actor_id
end
end
In the example above, we define:
movies
: Stores information about movies, including an auto-incrementing id
and a title
.
screenplays
: Stores the screenplay contents for movies, linking to the movies
table with movie_id
.
actors
: Stores actor names, with an auto-incrementing id
.
directors
: Stores director names, associated with a movie via movie_id
.
movies_actors
: A join table to link movies
and actors
, establishing a many-to-many relationship between movies and actors.
Once the schema has been defined, the next step is to initialize the database. This is done by calling the Schema.init
method (or in our case, AcmeDB2.init
), which creates the tables based on the schema.
Set up the environment: Ensure that the DATABASE_URL
environment variable is correctly set to point to your database connection string (for PostgreSQL in this case).
Call Schema.init
: After defining the schema, you initialize the database by calling the init
method on the schema object.
AcmeDB2.init
This command creates all the tables and applies the structure you defined in the schema to your actual database.
# db_context the schema
AcmeDB2 = CQL::Schema.define(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :movies do
primary :id, Int64, auto_increment: true
text :title
end
table :screenplays do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :content
end
table :actors do
primary :id, Int64, auto_increment: true
text :name
end
table :directors do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :name
end
table :movies_actors do
primary :id, Int64, auto_increment: true
bigint :movie_id
bigint :actor_id
end
end
# Initialize the database
AcmeDB2.init
When AcmeDB2.init
is called, the following happens:
The database connection is established using the URI provided in the schema (e.g., the PostgreSQL database connection).
CQL creates the tables (movies
, screenplays
, actors
, directors
, and movies_actors
) in the database if they don't already exist.
Primary keys, relationships, and any constraints are applied as defined in the schema.
After calling AcmeDB2.init
, you can verify the tables are created in your database by using a PostgreSQL client or a GUI tool such as pgAdmin
. You should see the tables with their respective columns and relationships as defined in the schema.
-- Example SQL to verify the tables were created
SELECT table_name FROM information_schema.tables WHERE table_schema = 'public';
-- To check the structure of a table, use:
\d movies -- This shows the structure of the 'movies' table
Initializing the database after schema creation is a simple process with CQL. After defining your tables and relationships using CQL::Schema.define
, you can call the init
method to apply your schema to the actual database.
This method ensures that your database is correctly structured and ready to use, allowing you to focus on developing the application logic without worrying about manual database setup.
AcmeDB2.init # Initializes the schema and sets up the database
The CQL::Insert
class in the CQL (Crystal Query Language) module is a powerful tool designed to simplify the process of inserting records into a database. As a software developer, you'll find this class essential for building INSERT
queries in a clean, readable, and chainable way.
In this guide, we’ll walk through the core functionality, explain each method in detail, and provide real-world examples to help you understand how to integrate CQL::Insert
into your applications.
Insert records into any table with ease.
Insert multiple records in a single query.
Insert records using data fetched from another query.
Get the last inserted ID after an insert.
Chainable, intuitive syntax for building complex queries.
Let’s start with a simple example of inserting a new user into the users
table.
insert
.into(:users)
.values(name: "Alice", email: "alice@example.com", age: 29)
.commit
This example demonstrates how you can insert a new user with their name, email, and age using the values
method, followed by commit
to execute the insert.
Let's dive into the individual methods and how you can use them.
into(table : Symbol)
Purpose: Specifies the table into which the data will be inserted. This is your starting point for any insert operation.
Parameter: table
(Symbol) — The name of the table to insert into.
Returns: Insert
object (enabling chaining).
Example:
insert.into(:users)
In the above code, we're targeting the users
table. This is where subsequent data will be inserted.
values(**fields)
Purpose: Specifies the data (fields and values) to insert. The values
method can accept either a hash or keyword arguments.
Parameter: fields
(Hash or keyword arguments) — A mapping of column names to their values.
Returns: Insert
object (for chaining).
Real-World Example 1: Adding a Single User
insert
.into(:users)
.values(name: "Bob", email: "bob@example.com", age: 35)
.commit
Here, we’re adding a new user named Bob, specifying their name
, email
, and age
.
Real-World Example 2: Adding Multiple Users in One Query
insert
.into(:users)
.values([
{name: "John", email: "john@example.com", age: 30},
{name: "Jane", email: "jane@example.com", age: 25}
]).commit
This example demonstrates how you can insert multiple users in a single query. It’s efficient and reduces database round trips.
last_insert_id(as type : PrimaryKeyType = Int64)
Purpose: Retrieves the ID of the last inserted row. This is incredibly useful when you need to work with the inserted record immediately after an insert, especially in cases where the primary key is automatically generated.
Parameter: type
(default: Int64
) — The data type of the returned ID.
Returns: The last inserted ID as Int64
or the specified type.
Example:
last_id = insert
.into(:users)
.values(name: "Charlie", email: "charlie@example.com", age: 22)
.last_insert_id
puts last_id # Outputs the last inserted ID
query(query : Query)
Purpose: Instead of manually specifying values, you can use data fetched from another query to populate the insert. This is useful in situations like copying data from one table to another.
Parameter: query
— A query object that fetches the data to insert.
Returns: Insert
object (for chaining).
Real-World Example: Copying Data from One Table to Another
Imagine you want to copy user data from an archive table (archived_users
) to the main users
table.
insert
.into(:users)
.query(select.from(:archived_users).where(active: true))
.commit
In this example, we’re selecting all active users from archived_users
and inserting them into the users
table in one go.
back(*columns : Symbol)
Purpose: After an insert operation, you may want to return certain columns, like an ID or timestamp. The back
method allows you to specify which columns should be returned.
Parameter: columns
— One or more symbols representing the columns to return.
Returns: Insert
object (for chaining).
Example:
insert
.into(:users)
.values(name: "David", email: "david@example.com", age: 28)
.back(:id)
.commit
In this case, after inserting the new user, we’re returning the id
of the newly inserted row.
commit
Purpose: Executes the built INSERT
query and commits the transaction to the database. This is the final step in your insert operation.
Returns: The result of the insert, typically the number of affected rows or the last inserted ID.
Example:
insert
.into(:users)
.values(name: "Eva", email: "eva@example.com", age: 31)
.commit
This will execute the INSERT
statement and commit it to the database, saving the new user’s data.
Let’s consider a more advanced example where you insert a new user, return their ID, and use it in subsequent operations.
user_id = insert
.into(:users)
.values(name: "Frank", email: "frank@example.com", age: 27)
.last_insert_id
# Now use `user_id` in another query, e.g., to insert into a related table
insert
.into(:user_profiles)
.values(user_id: user_id, bio: "Software developer from NYC", twitter: "@frankdev")
.commit
In this example, after inserting the new user, we immediately get the user_id
and use it to insert data into the user_profiles
table.
In case an insert operation fails, the commit
method automatically logs the error and raises an exception. This allows you to catch and handle the error as needed in your application.
Example:
begin
insert
.into(:users)
.values(name: nil) # This will likely fail due to a NOT NULL constraint
.commit
rescue ex
puts "Insert failed: #{ex.message}"
end
The CQL::Insert
class provides a flexible, chainable interface for inserting data into a database, whether you're inserting a single record, multiple records, or even records based on the results of a query. Its intuitive API makes it easy to use in real-world applications, and with error handling built-in, it helps ensure robust database operations.
With its simple syntax and powerful features, CQL::Insert
streamlines your database interactions, allowing you to focus on building great features in your Crystal applications.
CQL (Crystal Query Language) is a powerful tool designed for developers working with the Crystal programming language. It provides a streamlined interface for interacting with SQL databases, combining the flexibility of raw SQL with the safety and clarity of Crystal’s type system. CQL simplifies complex database operations, making it easier to perform CRUD (Create, Read, Update, Delete) operations, manage database schemas, and execute advanced queries.
Database-agnostic: CQL supports multiple databases, ensuring flexibility in project development.
Active Record Pattern: Integrates seamlessly with Crystal structs, allowing developers to work with database records as native Crystal objects.
Query Builder: Provides an intuitive API for constructing complex SQL queries, including joins, subqueries, and transactions.
Associations: CQL supports defining relationships between tables (has_many
, belongs_to
, many_to_many
), making it easy to navigate related data.
Migrations: Facilitates schema evolution by allowing developers to create and manage database migrations.
CQL is designed to work with a range of SQL databases, including:
PostgreSQL: Full support with advanced features like JSONB, arrays, and full-text search.
MySQL: Support for common MySQL operations, though more advanced features may be limited.
SQLite: Lightweight database support for development and testing environments.
CQL is ideal for developers looking to integrate Crystal with SQL databases, providing a robust toolset for building data-driven applications. Whether you are developing a small-scale application or a large enterprise system, CQL offers the performance and scalability needed to manage your database operations efficiently.
CQL’s core concepts revolve around providing developers with tools to efficiently interact with databases through well-structured APIs. These concepts are fundamental to building and manipulating data models, performing queries, and managing transactions. Here's a high-level overview:
Schemas: Define the structure of the database, including tables, columns, and data types.
CRUD Operations: Simplify creating, reading, updating, and deleting records in the database.
Query Builder: Enables constructing SQL queries using Crystal code with features like joins and subqueries.
Transactions: Ensure safe and atomic execution of multiple database operations.
The CQL::Query
class is designed to simplify the creation and execution of SQL queries. By using a structured, chainable API, you can build complex SQL queries while maintaining clean and readable code.
This guide walks you through how to create, modify, and execute queries using real-world examples. We'll also explore various methods for selecting, filtering, joining, and ordering data, with a focus on practical use cases.
Select columns and filter records using simple, chainable methods.
Join tables for complex queries involving multiple relationships.
Aggregate data using functions like COUNT
, SUM
, and AVG
.
Order and limit your result sets for more precise control.
Fetch results as objects or raw data for immediate use in your application.
Let's begin by selecting user data from a users
table:
query = CQL::Query.new(schema)
query.select(:name, :age).from(:users).where(name: "Alice").all(User)
This query will return all users with the name "Alice", casting the result to the User
type.
Below is a breakdown of the key methods in the CQL::Query
class and how you can use them in your applications.
select(*columns : Symbol)
Purpose: Specifies the columns to select in the query.
Parameters: columns
— One or more symbols representing the columns you want to select.
Returns: Query
object for chaining.
Real-World Example: Selecting Columns
query = CQL::Query.new(schema)
query.select(:name, :email).from(:users)
This query selects the name
and email
columns from the users
table.
from(*tables : Symbol)
Purpose: Specifies the tables to query from.
Parameters: tables
— One or more symbols representing the tables to query from.
Returns: Query
object for chaining.
Real-World Example: Querying a Table
query.from(:users)
This query selects from the users
table.
where(hash : Hash(Symbol, DB::Any))
Purpose: Adds filtering conditions to the query.
Parameters: hash
— A key-value hash representing the column and its corresponding value for the WHERE
clause.
Returns: Query
object for chaining.
Real-World Example: Filtering Data
query.from(:users).where(name: "Alice", age: 30)
This will generate a query with the WHERE
clause: WHERE name = 'Alice' AND age = 30
.
all(as : Type)
Purpose: Executes the query and returns all matching results, casting them to the specified type.
Parameters: as
— The type to cast the results to.
Returns: An array of the specified type.
Real-World Example: Fetching All Results
users = query.select(:name, :email).from(:users).all(User)
This will return all users as an array of User
objects.
first(as : Type)
Purpose: Executes the query and returns the first matching result, casting it to the specified type.
Parameters: as
— The type to cast the result to.
Returns: The first matching result of the query.
Real-World Example: Fetching the First Result
user = query.select(:name, :email).from(:users).where(name: "Alice").first(User)
This returns the first user with the name "Alice".
count(column : Symbol = :*)
Purpose: Adds a COUNT
aggregate function to the query, counting the specified column.
Parameters: column
— The column to count (default is *
, meaning all rows).
Returns: Query
object for chaining.
Real-World Example: Counting Rows
count = query.from(:users).count(:id).get(Int64)
This will count the number of rows in the users
table and return the result as an Int64
.
join(table : Symbol, on : Hash)
Purpose: Adds a JOIN
clause to the query, specifying the table and the condition for joining.
Parameters:
table
: The table to join.
on
: A hash representing the join condition, mapping columns from one table to another.
Returns: Query
object for chaining.
Real-World Example: Joining Tables
query
.from(:users)
.join(:orders, on: {users.id => orders.user_id})
This query joins the users
table with the orders
table on the condition that users.id
equals orders.user_id
.
order(*columns : Symbol)
Purpose: Specifies the columns by which to order the results.
Parameters: columns
— The columns to order by.
Returns: Query
object for chaining.
Real-World Example: Ordering Results
query.from(:users).order(:name, :age)
This orders the query results by name
first and then by age
.
limit(value : Int32)
Purpose: Limits the number of rows returned by the query.
Parameters: value
— The number of rows to return.
Returns: Query
object for chaining.
Real-World Example: Limiting Results
query.from(:users).limit(10)
This limits the query to return only the first 10 rows.
get(as : Type)
Purpose: Executes the query and returns a scalar value, such as the result of an aggregate function (e.g., COUNT
, SUM
).
Parameters: as
— The type to cast the result to.
Returns: The scalar result of the query.
Real-World Example: Getting a Scalar Value
total_users = query.from(:users).count(:id).get(Int64)
This returns the total number of users as an Int64
.
each(as : Type, &block)
Purpose: Iterates over each result, yielding each row to the provided block.
Parameters:
as
: The type to cast each row to.
&block
: The block to execute for each row.
Returns: Nothing (used for iteration).
Real-World Example: Iterating Over Results
query.from(:users).each(User) do |user|
puts user.name
end
This will print the name of each user in the users
table.
distinct
Purpose: Sets the DISTINCT
flag to return only unique rows.
Returns: Query
object for chaining.
Real-World Example: Fetching Distinct Results
query.from(:users).distinct
This will generate a query that returns only distinct rows from the users
table.
Let's create a real-world example that combines several methods. Suppose you want to fetch the first 5 users who have placed an order, ordered by their name, and return the result as User
objects:
query = CQL::Query.new(schema)
users = query
.select(:name, :email)
.from(:users)
.join(:orders, on: {users.id => orders.user_id})
.where(active: true)
.order(:name)
.limit(5)
.all(User)
users.each do |user|
puts "Name: #{user.name}, Email: #{user.email}"
end
In this query:
We select the name
and email
columns from users
.
We join the orders
table to ensure the user has placed an order.
We filter for active users (where(active: true)
).
We order the results by name
.
We limit the results to 5 users.
The CQL::Query
class offers a flexible and intuitive API for building and executing SQL queries in your Crystal applications. With methods for selecting, joining, filtering, and aggregating data, you can handle even the most complex queries with ease.
Whether you're building basic queries or handling complex database operations, the CQL::Query
class provides the tools you need to write clean, efficient, and maintainable code.
The CQL::AlterTable
class in the CQL framework provides a structured way to make schema changes to your database tables. It allows you to perform actions like adding, dropping, and renaming columns, as well as adding foreign keys, renaming tables, and creating indexes. This guide walks you through the most common use cases with real-world examples that software developers can apply in their projects.
CQL::AlterTable
?When your application evolves, you often need to modify your database structure. CQL::AlterTable
lets you:
Add new columns to accommodate growing data needs.
Remove or rename columns as your data model refines.
Enforce foreign key relationships and maintain referential integrity.
Create or remove indexes for performance tuning.
Let’s start with a basic example where we modify the users
table by adding a new column email
, removing the age
column, and renaming the email
column to user_email
.
Given the AcmeDB2 schema with a Users table:
AcmeDB2 = CQL::Schema.build(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :users do
primary
text name, size: 150
end
end
alter = AlterTable.new(users_table, schema)
AcmeDB.alter :users do
add_column(:email, "string", null: false, unique: true)
drop_column(:age)
rename_column(:email, :user_email)
end
This example:
Adds a new email
column that cannot be NULL
and must be unique.
Drops the age
column, removing it from the table.
Renames the email
column to user_email
.
CQL::AlterTable
add_column(name : Symbol, type : Any, **options)
Purpose: Adds a new column to the table with flexible options for setting the column's properties, such as type, default value, uniqueness, etc.
Parameters:
name
: The name of the column to add.
type
: The data type of the column (e.g., String
, Int32
).
Additional options:
null
: Whether the column can be NULL
(default: true
).
default
: The default value for the column.
unique
: Whether the column should have a unique constraint (default: false
).
size
: Optionally specify the size of the column (for strings or numbers).
index
: Whether the column should be indexed (default: false
).
Real-World Example: Adding an Email Column
alter.add_column(:email, "string", null: false, unique: true)
This adds an email
column to the table, ensures it is NOT NULL
, and enforces a uniqueness constraint.
drop_column(name : Symbol)
Purpose: Removes an existing column from the table.
Parameters:
name
: The name of the column to drop.
Real-World Example: Dropping a Column
alter.drop_column(:age)
This removes the age
column from the table.
rename_column(old_name : Symbol, new_name : Symbol)
Purpose: Renames an existing column in the table.
Parameters:
old_name
: The current name of the column.
new_name
: The new name for the column.
Real-World Example: Renaming a Column
alter.rename_column(:email, :user_email)
This renames the email
column to user_email
.
change_column(name : Symbol, type : Any)
Purpose: Changes the data type of an existing column.
Parameters:
name
: The name of the column.
type
: The new data type for the column.
Real-World Example: Changing a Column’s Type
alter.change_column(:age, "string")
This changes the age
column’s type from whatever it was (likely an Int32
) to a string
.
rename_table(new_name : Symbol)
Purpose: Renames the entire table.
Parameters:
new_name
: The new name for the table.
Real-World Example: Renaming a Table
alter.rename_table(:customers)
This renames the table from users
to customers
.
foreign_key(name : Symbol, columns : Array(Symbol), table : Symbol, references : Array(Symbol), **options)
Purpose: Adds a foreign key constraint to the table.
Parameters:
name
: The name of the foreign key.
columns
: The columns in the current table to use as the foreign key.
table
: The referenced table.
references
: The columns in the referenced table.
Options:
on_delete
: Action to take on delete (default: NO ACTION
).
on_update
: Action to take on update (default: NO ACTION
).
Real-World Example: Adding a Foreign Key
alter.foreign_key(:fk_movie_id, [:movie_id], :movies, [:id], on_delete: "CASCADE")
This adds a foreign key fk_movie_id
on the movie_id
column, linking it to the id
column in the movies
table. On delete, it cascades the delete.
drop_foreign_key(name : Symbol)
Purpose: Removes a foreign key constraint from the table.
Parameters:
name
: The name of the foreign key to remove.
Real-World Example: Dropping a Foreign Key
alter.drop_foreign_key(:fk_movie_id)
This drops the foreign key constraint fk_movie_id
from the table.
create_index(name : Symbol, columns : Array(Symbol), unique : Bool = false)
Purpose: Adds an index to the specified columns.
Parameters:
name
: The name of the index.
columns
: The columns to index.
unique
: Whether the index should enforce uniqueness (default: false
).
Real-World Example: Creating a Unique Index
alter.create_index(:index_users_on_email, [:email], unique: true)
This creates a unique index on the email
column, ensuring that email addresses are unique across the table.
drop_index(name : Symbol)
Purpose: Removes an index from the table.
Parameters:
name
: The name of the index to remove.
Real-World Example: Dropping an Index
alter.drop_index(:index_users_on_email)
This drops the index index_users_on_email
from the table.
to_sql(visitor : Expression::Visitor)
Purpose: Generates the SQL string corresponding to the current set of table alterations.
Parameters:
visitor
: The SQL generator that converts the actions to SQL.
Real-World Example: Generating SQL for Table Alterations
sql = alter.to_sql(visitor)
puts sql
This will generate the SQL statements that correspond to the actions taken (e.g., adding columns, dropping columns).
Here’s an advanced example where we modify the users
table by adding and removing columns, renaming the table, and creating an index:
alter = AlterTable.new(users_table, schema)
sql = AcmeDB2.alter :users do
add_column(:email, "string", null: false, unique: true)
drop_column(:age)
rename_table(:customers)
create_index(:index_customers_on_email, [:email], unique: true)
end.to_sql
puts sql
This code:
Adds a email
column with NOT NULL
and UNIQUE
constraints.
Drops the age
column.
Renames the users
table to customers
.
Creates a unique index on the email
column.
Generates the SQL statements that implement these actions.
The CQL::AlterTable
class provides a simple and flexible interface for modifying your database schema. With its chainable methods, you can easily add, drop, or modify columns, rename tables, and manage foreign keys and indexes. This makes managing your database schema effortless, allowing you to focus on building robust, scalable applications.
Database migrations are essential for managing changes to your schema over time in a controlled manner. In CQL, migrations are handled through the Migration
and Migrator
classes. This guide will help you understand how to create, apply, rollback, and manage migrations using CQL::Migrator
in your projects.
Migrations allow you to:
Apply changes to your database schema over time.
Roll back changes in case of errors or updates.
Track applied and pending changes, ensuring consistency across environments.
Let’s start with a simple example. Suppose we need to add a users
table to our database with two columns: name
and age
.
The up
method: defines the changes to apply when the migration is run (e.g., adding new columns).
The down
method: defines how to revert the changes (e.g., dropping columns).
Versioning: Each migration is assigned a version number, which ensures migrations are run in the correct order.
Before applying migrations, you need to set up the schema and create an instance of the Migrator
.
The migrator, upon initialization, automatically creates a schema_migrations
table to track which migrations have been applied.
To apply all pending migrations, simply call the up
method on the migrator
object:
This will apply all pending migrations in order of their version numbers.
Applying Migrations Up to a Specific Version
You can also apply migrations up to a specific version:
This will apply all migrations up to version 1_i64
.
To roll back the last migration, use the down
method:
You can also roll back to a specific migration version:
This rolls back all migrations down to version 1_i64
.
If you want to rollback and then re-apply the last migration, use the redo
method:
This first rolls back the last migration and then re-applies it.
You can list applied, pending, and rolled-back migrations with the following commands:
List Applied Migrations:
List Pending Migrations:
List Rolled Back Migrations:
These commands provide a clear view of the current state of your migrations, making it easy to track progress and issues.
Checking the Last Applied Migration
You can retrieve information about the last applied migration using:
This gives you details about the last migration that was successfully applied.
Here’s an example where we define multiple migrations and apply them sequentially:
Versioning ensures that migrations are applied in the correct order.
Each migration can be applied and rolled back independently, offering flexibility in managing your database schema.
The CQL::Migrator
class makes it easy to manage database migrations in a structured and version-controlled manner. By following this guide, you can:
Create and apply migrations to modify your schema.
Roll back changes if needed.
Track applied and pending migrations to keep your database consistent across environments.
This approach is essential for teams working on large applications where database changes need to be applied safely and consistently over time.
class CreateUsersTable < CQL::Migration(1)
def up
schema.alter :users do
add_column :name, String
add_column :age, Int32
end
end
def down
schema.alter :users do
drop_column :name
drop_column :age
end
end
end
schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
...
end
migrator = CQL::Migrator.new(schema)
migrator.up
migrator.up_to(1_i64)
migrator.down
migrator.down_to(1_i64)
migrator.redo
migrator.print_applied_migrations
migrator.print_pending_migrations
migrator.print_rolled_back_migrations
last_migration = migrator.last
puts last_migration
class CreateMoviesTable < CQL::Migration(2)
def up
schema.alter :movies do
add_column :title, String
add_column :release_year, Int32
end
end
def down
schema.alter :movies do
drop_column :title
drop_column :release_year
end
end
end
class CreateActorsTable < CQL::Migration
self.version = 3_i64
def up
schema.alter :actors do
add_column :name, String
end
end
def down
schema.alter :actors do
drop_column :name
end
end
end
# Apply the migrations
migrator.up
The CQL::Delete
class provides a structured and flexible way to build and execute SQL DELETE
queries in your Crystal applications. This guide will help you understand how to create delete queries, apply conditions, and execute them to remove records from your database.
Delete records from any table in a straightforward manner.
Filter records to delete using flexible WHERE
conditions.
Return columns after deletion if needed.
Chainable syntax for clean and maintainable query building.
Let’s start with a simple example of deleting a user from the users
table where the id
is 1.
delete = CQL::Delete.new(schema)
.from(:users)
.where(id: 1)
.commit
This query deletes the record in the users
table where id = 1
.
The following section provides a breakdown of the key methods available in the CQL::Delete
class and how to use them effectively.
from(table : Symbol)
Purpose: Specifies the table from which records will be deleted.
Parameters: table
— A symbol representing the table name.
Returns: Delete
object (for chaining).
Real-World Example: Specifying the Table
delete.from(:users)
This sets the users
table as the target for the delete operation.
where(**fields)
Purpose: Adds a WHERE
clause to filter the records to be deleted.
Parameters: fields
— A key-value hash where keys represent column names and values represent the conditions to match.
Returns: Delete
object (for chaining).
Real-World Example: Filtering by Conditions
delete
.from(:users)
.where(id: 1)
This filters the query to only delete the user where id = 1
.
where(&block)
Purpose: Adds a WHERE
clause using a block for more complex filtering conditions.
Parameters: A block that defines the filtering logic using a filter builder.
Returns: Delete
object (for chaining).
Real-World Example: Using a Block for Conditions
delete
.from(:users)
.where { |w| w.age < 30 }
This deletes all users where the age
is less than 30.
commit
Purpose: Executes the delete query and commits the changes to the database.
Returns: A DB::Result
object representing the result of the query execution.
Real-World Example: Committing the Delete
delete = CQL::Delete.new(schema)
.from(:users)
.where(id: 1)
.commit
This deletes the user from the users
table where id = 1
and commits the change.
using(table : Symbol)
Purpose: Adds a USING
clause to the delete query, useful when deleting records based on conditions from another table.
Parameters: table
— A symbol representing the name of the table to use in the USING
clause.
Returns: Delete
object (for chaining).
Real-World Example: Using Another Table for Deletion
delete
.from(:users)
.using(:posts)
.where { |w| w.posts.user_id == w.users.id }
This example deletes users where they are linked to posts based on the condition posts.user_id = users.id
.
back(*columns : Symbol)
Purpose: Specifies the columns to return after the delete operation.
Parameters: columns
— An array of symbols representing the columns to return.
Returns: Delete
object (for chaining).
Real-World Example: Returning Columns After Deletion
delete
.from(:users)
.where(id: 1)
.back(:name, :email)
.commit
This deletes the user with id = 1
and returns the name
and email
of the deleted record.
to_sql(gen = @schema.gen)
Purpose: Generates the SQL query and parameters required for the delete operation.
Parameters: gen
— The generator used for SQL generation (default: schema generator).
Returns: A tuple containing the SQL query string and the parameters.
Real-World Example: Generating SQL for Deletion
delete = CQL::Delete.new(schema)
.from(:users)
.where(id: 1)
sql, params = delete.to_sql
puts sql # "DELETE FROM users WHERE id = $1"
puts params # [1]
This generates the raw SQL query and its associated parameters without executing it.
Let’s combine multiple methods to handle a more advanced use case. Suppose you want to delete a user from the users
table where they have no associated posts, and you want to return the deleted user’s name and email:
delete = CQL::Delete.new(schema)
result = delete
.from(:users)
.using(:posts)
.where { |w| w.posts.user_id == w.users.id && w.posts.id.nil? }
.back(:name, :email)
.commit
puts result # This returns the name and email of the deleted user(s).
In this query:
We specify the users
table as the target for deletion.
We use the posts
table to filter users without any posts.
We return the name
and email
of the deleted user(s).
The CQL::Delete
class provides an intuitive and powerful interface for deleting records in your Crystal applications. With chainable methods for setting conditions, joining tables, and selecting return columns, you can easily construct and execute delete queries with precision and clarity.
Whether you need to delete specific records or perform complex, condition-based deletions, the CQL::Delete
class ensures that your queries are efficient and maintainable.
Active Record is a design pattern used in Object-Relational Mapping (ORM) that simplifies database access by linking database tables directly to classes in your application. Each class represents a table, and each instance of the class corresponds to a row in that table. Active Record makes it easy to perform CRUD (Create, Read, Update, Delete) operations on the database.
In the context of CQL (Crystal Query Language) and Crystal, the Active Record pattern can be implemented by leveraging the object-oriented nature of Crystal and the querying capabilities provided by CQL. Here's how you can think about the Active Record pattern using CQL:
Table-Class Mapping: Each class corresponds to a table in the database.
Row-Object Mapping: Each object is an instance of a class and corresponds to a row in the database table.
Database Operations as Methods: Methods on the object handle database interactions (e.g., .save
, .delete
, .find
).
Associations: Relationships between tables (e.g., belongs_to
, has_many
) are handled within the class structure.
Validation: Logic to ensure data integrity is embedded within the class.
In Active Record, a model is a class that represents a database table. The class will contain attributes (columns), methods for interacting with the data, and associations.
Here’s an example of a User
model:
struct User
include CQL::ActiveRecord::Model(Int32)
# Schame name AcmeDB, table name :users
db_context AcmeDB, :users
property id : Int32?
property name : String
property email : String
property created_at : Time
property updated_at : Time
end
In this example:
struct User
is used instead of class User
Include CQL::Record(User, Int32)
specifies that User
is the model and the primary key is of type Int32
.
The model still contains the properties (id
, name
, email
, created_at
, updated_at
), but now we delegate all Active Record-like operations (e.g., save
, delete
) to CQL::Record
.
In the Active Record pattern, CRUD operations (Create, Read, Update, Delete) are performed directly on the class and instance methods. Here’s how you can implement CRUD with CQL:
To create a new record in the database, instantiate a new object of the model class and call .save
to persist it:
user = User.new
user.name = "John Doe"
user.email = "john.doe@example.com"
user.created_at = Time.now
user.updated_at = Time.now
user.save # INSERTS the new user into the database
This will generate an INSERT INTO
SQL statement and persist the user in the users
table.
To retrieve records from the database, you can use class-level methods like .find
or .all
. For example:
Fetch all users:
users = User.all # SELECT * FROM users
Find a user by id
:
user = User.find(1) # SELECT * FROM users WHERE id = 1
To update an existing record, modify the object and call .save
again. This will generate an UPDATE
SQL statement:
user = User.find(1)
user.name = "Jane Doe"
user.save # UPDATE users SET name = 'Jane Doe' WHERE id = 1
To delete a record, find the object and call .delete
:
user = User.find(1)
user.delete # DELETE FROM users WHERE id = 1
Active Record also simplifies relationships between tables, such as has_many
and belongs_to
. In CQL, you can implement these relationships like this:
A User
has many Posts
. You can db_context the association like this:
struct User < CQL::Record(User, Int32)
property id : Int32?
property name : String
property email : String
property created_at : Time
property updated_at : Time
has_many :posts, Post
end
struct Post
include CQL::ActiveRecord::Model(Int32)
property id : Int32?
property title : String
property body : String
property created_at : Time
property updated_at : Time
belongs_to :user, User
end
Now you can fetch the posts for a user:
user = User.find(1)
posts = user.posts # SELECT * FROM posts WHERE user_id = 1
The Post
class has a belongs_to
relationship with User
. This means each post belongs to a user:
post = Post.find(1)
user = post.user # SELECT * FROM users WHERE id = post.user_id
Here is a summary of the collection methods provided in the CQL::Relations::Collection
and CQL::Relations::ManyCollection
classes for managing associations in a one-to-many and many-to-many relationship in CQL:
all
:
Returns all associated records for the parent record.
Example: movie.actors.all
reload
:
Reloads the associated records from the database.
Example: movie.actors.reload
ids
:
Returns a list of primary keys for the associated records.
Example: movie.actors.ids
<<
:
Adds a new record to the association and persists it to the database.
Example: movie.actors << Actor.new(name: "Laurence Fishburne")
empty?
:
Checks if the association has any records.
Example: movie.actors.empty?
**exists?(**attributes)
**:
Checks if any associated records exist that match the given attributes.
Example: movie.actors.exists?(name: "Keanu Reeves")
size
:
Returns the number of associated records.
Example: movie.actors.size
**find(**attributes)
**:
Finds associated records that match the given attributes.
Example: movie.actors.find(name: "Keanu Reeves")
**create(**attributes)
**:
Creates a new record with the provided attributes and associates it with the parent.
Example: movie.actors.create(name: "Carrie-Anne Moss")
create!(record)
:
Creates and persists a new record with the provided attributes, raising an error if it fails.
Example: movie.actors.create!(name: "Hugo Weaving")
ids=(ids : Array(Pk))
:
Associates the parent record with the records that match the provided primary keys.
Example: movie.actors.ids = [1, 2, 3]
delete(record : Target)
:
Deletes the associated record from the parent record if it exists.
Example: movie.actors.delete(Actor.find(1))
delete(id : Pk)
:
Deletes the associated record by primary key.
Example: movie.actors.delete(1)
clear
:
Removes all associated records for the parent record.
Example: movie.actors.clear
In addition to the methods inherited from Collection
, the ManyCollection
class also manages associations through a join table in many-to-many relationships.
create(record : Target)
:
Associates the parent record with the created record through a join table.
Example: movie.actors.create!(name: "Carrie-Anne Moss")
delete(record : Target)
:
Deletes the association through the join table between the parent and associated record.
Example: movie.actors.delete(Actor.find(1))
ids=(ids : Array(Pk))
:
Associates the parent record with the records matching the primary keys through the join table.
Example: movie.actors.ids = [1, 2, 3]
These methods provide powerful ways to interact with and manage associations between records in a CQL-based application using both one-to-many and many-to-many relationships.
Active Record often includes validations to ensure that data meets certain criteria before saving. In CQL, you can add custom validation logic inside the class:
struct User
include CQL::ActiveRecord::Model(Int32)
property id : Int32?
property name : String
property email : String
property created_at : Time
property updated_at : Time
def validate
raise "Name can't be empty" if name.empty?
raise "Email can't be empty" if email.empty?
end
def save
validate
super # Calls the original save method provided by CQL::Entity
end
end
In this example, before saving a user, the validate
method is called to ensure that the name and email are not empty.
Although not strictly part of Active Record, migrations are commonly used with it to modify database schemas over time. In CQL, migrations can be written using a migration system to create and alter tables. For example:
class CreateUsersTable < CQL::Migration
self.version = 1_i64
def up
schema.alter :users do
add_column :name, String
add_column :age, Int32
end
end
def down
schema.alter :users do
drop_column :name
drop_column :age
end
end
end
This migration would create the users
table with the specified columns.
Model Representation: Each class (like User
, Post
) maps directly to a database table.
CRUD Operations: Operations like .save
, .delete
, .find
, and .all
are built into the CQL framework, allowing for seamless interaction with the database.
Associations: Relationships between models are defined using macros like has_many
and belongs_to
, which make querying associated records straightforward.
Encapsulation of Business Logic: Validation and other business rules can be embedded directly into model classes.
Database Migrations: Schema changes are managed through migrations, which help keep the database structure synchronized with the application's models.
db_context your models:
Create User
and Post
classes that correspond to users
and posts
tables.
Run Migrations:
Use migrations to create or modify the database schema.
Perform CRUD operations:
Create, read, update, and delete records using model methods like .save
and .delete
.
Manage relationships:
db_context associations like has_many
and belongs_to
to handle relationships between models.
Enforce business rules:
Use validation methods to ensure data integrity.
By following the Active Record pattern in CQL, you can build a robust data access layer in your Crystal application with minimal effort while keeping your code clean and maintainable.
The Repository pattern is a design pattern that mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects. It centralizes data access logic, promoting a cleaner separation of concerns and making the application easier to test and maintain.
In CQL (Crystal Query Language), while the Active Record pattern is a prominent feature, CQL's design is flexible enough to support the Repository pattern. This allows developers to abstract the underlying data persistence mechanisms further and work with domain objects in a more decoupled manner.
Abstraction of Data Access: The repository provides an abstraction layer over data storage. Consumers of the repository work with domain objects and are unaware of how those objects are persisted or retrieved.
Collection-Like Interface: Repositories often expose methods that resemble collection operations, such as add
, remove
, find
, all
, etc.
Centralized Query Logic: Queries related to a specific aggregate root or entity are encapsulated within its repository.
Decoupling: It decouples the domain model from the data access concerns, improving testability and maintainability.
Unit of Work (Optional but Common): Repositories are often used in conjunction with a Unit of Work pattern to manage transactions and track changes to objects.
CQL provides a generic CQL::Repository(T, Pk)
class that can be used as a base for creating specific repositories for your domain entities.
First, you need a model (often a plain Crystal struct or class) that represents your domain entity. Unlike Active Record models, these entities typically don't include persistence logic themselves.
# Example: A simple User entity
struct User
property id : Int32?
property name : String
property email : String
def initialize(@name : String, @email : String, @id : Int32? = nil)
end
end
You create a repository for your entity by inheriting from CQL::Repository(T, Pk)
, where T
is your entity type and Pk
is the type of its primary key.
require "cql"
# Define your schema (database connection)
MySchema = CQL::Schema.define(
:my_app_db,
adapter: CQL::Adapter::Postgres, # or any other supported adapter
uri: ENV["DATABASE_URL"]? || "postgres://localhost:5432/my_app_dev"
)
# Define the User entity (as above)
struct User
property id : Int32?
property name : String
property email : String
def initialize(@name : String, @email : String, @id : Int32? = nil)
end
# Constructor to map from a Hash, often useful with database results
def self.new(attrs : Hash(Symbol, DB::Any))
new(
name: attrs[:name].as(String),
email: attrs[:email].as(String),
id: attrs[:id]?.as?(Int32)
)
end
end
# UserRepository inheriting from CQL::Repository
class UserRepository < CQL::Repository(User, Int32)
# The constructor takes the schema instance and the table name
def initialize(schema : CQL::Schema = MySchema, table_name : Symbol = :users)
super(schema, table_name)
end
# You can add custom query methods here
def find_active_users
query.where { active == true }.all(User)
end
def find_by_email_domain(domain : String)
query.where { email.like("%@#{domain}") }.all(User)
end
end
Explanation:
UserRepository < CQL::Repository(User, Int32)
: Defines a repository for User
entities where the primary key is Int32
.
initialize(schema : CQL::Schema = MySchema, table_name : Symbol = :users)
: The constructor expects a CQL::Schema
instance and the symbol representing the database table name (e.g., :users
).
super(schema, table_name)
: Calls the constructor of the base CQL::Repository
class.
find_active_users
, find_by_email_domain
: Custom methods that encapsulate specific query logic using CQL's query builder (query.where(...)
). The query
method is provided by the base CQL::Repository
.
Once defined, you can use the repository to interact with your data:
# Initialize the repository
user_repo = UserRepository.new
# Create a new user
# The `create` method in the base Repository expects a Hash of attributes
new_user_id = user_repo.create({name: "Alice Wonderland", email: "alice@example.com"})
puts "Created user with ID: #{new_user_id}"
# Find a user by ID
alice = user_repo.find(new_user_id.as(Int32))
if alice
puts "Found user: #{alice.name}"
else
puts "User not found."
end
# Fetch all users
all_users = user_repo.all
puts "All users:"
all_users.each do |user|
puts "- #{user.name} (#{user.email})"
end
# Update a user
if alice
user_repo.update(alice.id.not_nil!, {email: "alice.wonderland@newdomain.com"})
updated_alice = user_repo.find(alice.id.not_nil!)
puts "Updated email: #{updated_alice.try &.email}"
end
# Use custom repository methods
active_users = user_repo.find_active_users # Assuming an 'active' column and logic
puts "Active users: #{active_users.size}"
# Delete a user
if alice
user_repo.delete(alice.id.not_nil!)
puts "Deleted user #{alice.name}"
end
CQL::Repository(T, Pk)
The base CQL::Repository
class provides several convenient methods for common data operations:
query
: Returns a new CQL::Query
instance scoped to the repository's table.
insert
: Returns a new CQL::Insert
instance scoped to the repository's table.
update
: Returns a new CQL::Update
instance scoped to the repository's table.
delete
: Returns a new CQL::Delete
instance scoped to the repository's table.
build(attrs : Hash(Symbol, DB::Any)) : T
: Instantiates a new entity T
with the given attributes (does not persist).
all : Array(T)
: Fetches all records.
find(id : Pk) : T?
: Finds a record by its primary key, returns nil
if not found.
find!(id : Pk) : T
: Finds a record by its primary key, raises DB::NoResultsError
if not found.
**find_by(**fields) : T?
**: Finds the first record matching the given field-value pairs.
**find_all_by(**fields) : Array(T)
**: Finds all records matching the given field-value pairs.
create(attrs : Hash(Symbol, DB::Any)) : Pk
: Creates a new record with the given attributes and returns its primary key.
**create(**fields) : Pk
**: Creates a new record with the given attributes (using named arguments) and returns its primary key.
update(id : Pk, attrs : Hash(Symbol, DB::Any))
: Updates the record with the given id
using the provided attributes.
update(id : Pk, **fields)
: Updates the record with the given id
using the provided attributes (named arguments).
update_by(where_attrs : Hash(Symbol, DB::Any), update_attrs : Hash(Symbol, DB::Any))
: Updates records matching where_attrs
with update_attrs
.
update_all(attrs : Hash(Symbol, DB::Any))
: Updates all records in the table with the given attributes.
delete(id : Pk)
: Deletes the record with the given primary key.
**delete_by(**fields)
**: Deletes records matching the given field-value pairs.
delete_all
: Deletes all records in the table.
count : Int64
: Returns the total number of records in the table.
**exists?(**fields) : Bool
**: Checks if any records exist matching the given field-value pairs.
Improved Testability: You can easily mock repositories in your tests, isolating your domain logic from the database.
Centralized Data Access Logic: Keeps data access concerns separate from your domain entities and application services.
Flexibility: While CQL::Repository
provides a good starting point, you can customize it or even write your own repository implementations if needed, without altering your domain models significantly.
Clearer Intent: Queries are named and reside within the repository, making their purpose more explicit.
In larger applications where a clear separation between domain logic and data access is crucial.
When you need to support multiple data sources or switch data storage technologies with minimal impact on the domain layer.
When your querying logic becomes complex and you want to encapsulate it.
To improve the testability of your application by allowing easier mocking of data access.
While Active Record can be simpler for straightforward CRUD operations, the Repository pattern offers more structure and decoupling for complex applications, and CQL provides the tools to implement it effectively.
Defining the schema first is a fundamental approach in CQL, helping developers quickly structure their database while keeping their application’s data model in sync with real-world entities. By defining your schema upfront, you can rapidly iterate over your database tables, making it easy to adjust data structures as your application evolves. This method ensures that your schema is the single source of truth, giving you a clear view of how your data is organized and how relationships between different tables are modeled.
Faster Prototyping: With schemas defined at the outset, you can rapidly experiment with different table structures and relationships, making it easier to adjust your application’s data model without writing complex migrations from scratch.
Clear Data Structure: When your schema is predefined, the application’s data structure becomes clearer, allowing developers to conceptualize how data is organized and interact with tables more easily.
Consistency: Ensuring the schema matches the database at all times removes ambiguity when writing queries, handling relationships, or performing migrations.
Automatic Data Validation: CQL schemas enforce data types and constraints, such as primary
, auto_increment
, and text
, ensuring data integrity.
Simplified Query Building: Since the schema is explicit, writing queries becomes easier as you can reference schema objects directly in queries, avoiding mistakes or typos in table or column names.
Unlike traditional ORM libraries (e.g., Active Record in Rails or Ecto in Elixir), which often allow defining database models alongside the code and handling schema evolution through migrations, CQL encourages defining the database schema as the first step.
This "schema-first" approach differs from the "code-first" or "migration-based" methodologies in that it avoids relying on automatic migrations or conventions to infer the structure of the database. CQL enforces an explicit and structured approach to schema creation, ensuring the database schema reflects the actual architecture of your application.
Here’s a basic example of how to define a schema in CQL for a movie-related database:
Database name: :acme_db
defines the schema name.
Adapter: CQL::Adapter::Postgres
specifies the database adapter (in this case, PostgreSQL).
Connection URL: The uri: ENV["DATABASE_URL"]
specifies the database connection using environment variables.
Each table is explicitly defined with its columns, such as:
:movies
table has id
as the primary key and title
as a text
column.
:screenplays
, :actors
, and :directors
define relationships between movies and associated records.
This example shows how easy it is to define tables and manage relationships within the schema, leading to a more organized and coherent database structure that aligns with the application’s needs.
One significant advantage of CQL is the ability to define and manage multiple schemas within the same application. This is particularly useful in scenarios like multi-tenant applications, where each tenant or environment has a separate database schema. CQL makes switching between schemas seamless, enabling developers to organize different parts of the application independently while maintaining the same connection configuration.
This approach offers the following benefits:
Clear Separation of Data: Each schema can encapsulate its own set of tables and relationships, allowing better isolation and separation of concerns within the application. For example, you might have a main
schema for core business data and a separate analytics
schema for reporting.
Simple Switching: Switching between schemas is as simple as referring to the schema name, thanks to CQL’s structured definition of schemas. This allows dynamic switching at runtime, improving scalability in multi-tenant applications.
In this example, you define multiple schemas, and the application can easily switch between MainDB
and AnalyticsDB
depending on which database needs to be queried.
Improved Organization: Separate business logic data from other concerns like reporting, testing, or archiving.
Scalability: Ideal for multi-tenant applications, allowing each tenant to have its schema without interference.
By using CQL’s schema system, you gain not only speed and clarity in your database structure but also flexibility in scaling and organizing your application.
CQL Active Record models provide a rich set of methods for performing CRUD (Create, Read, Update, Delete) operations on your database records. These methods are designed to be intuitive and align with common Active Record patterns.
This guide assumes you have a model defined, for example:
There are several ways to create new records and persist them to the database.
new
+ save
/ save!
The most fundamental way is to create a new instance of your model using new
and then call save
or save!
to persist it.
save
: Attempts to save the record. Runs validations. If successful, populates the id
(if auto-generated) and returns true
. If validations fail or another before_save
callback halts the chain, it returns false
and does not persist the record. The errors
collection on the instance can be inspected.
save!
: Similar to save
, but raises an exception if the record cannot be saved.
Raises CQL::Errors::RecordInvalid
if validations fail.
May raise other database-related exceptions or CQL::Errors::RecordNotSaved
for other save failures.
create
/ create!
(Class Methods)These class methods provide a convenient way to instantiate and save a record in a single step.
Model.create(attributes)
: Creates a new instance with the given attributes (as a Hash or keyword arguments) and attempts to save
it. Returns the model instance (which might be invalid and not persisted if save
failed) or false
if a before_create
callback halted the process.
Model.create!(attributes)
: Similar to create
, but calls save!
internally. It will raise an exception (CQL::Errors::RecordInvalid
or other) if the record cannot be created and persisted.
find_or_create_by
(Class Method)This method attempts to find a record matching the given attributes. If found, it returns that record. If not found, it creates and persists a new record with those attributes (plus any additional ones provided if the find attributes are a subset of create attributes).
It uses create!
internally for the creation part, so it can raise exceptions if the creation fails (e.g., due to validations).
Summary of Common Finders:
Model.all
: Retrieves all records of the model type. Returns Array(Model)
.
Model.find?(id : Pk)
(or Model.find(id : Pk)
): Finds a record by its primary key. Returns Model?
(the instance or nil
).
Model.find!(id : Pk)
: Finds a record by its primary key. Returns Model
or raises DB::NoResultsError
(or similar if not found).
Model.find_by(**attributes)
: Finds the first record matching the given attributes. Returns Model?
.
Model.find_by!(**attributes)
: Finds the first record matching attributes. Returns Model
or raisesDB::NoResultsError
.
Model.find_all_by(**attributes)
: Finds all records matching attributes. Returns Array(Model)
.
Model.first
: Retrieves the first record (ordered by primary key). Returns Model?
.
Model.last
: Retrieves the last record (ordered by primary key). Returns Model?
.
Model.count
: Returns the total number of records as Int64
.
Model.query.[condition].count
: Counts records matching specific conditions.
To update existing records, you typically load an instance, modify its attributes, and then save it.
save
/ save!
This is the standard approach:
update
/ update!
(Instance Methods)These instance methods provide a shortcut to assign attributes and then save the record.
instance.update(attributes)
: Assigns the given attributes (Hash or keyword arguments) to the instance and then calls save
. Returns true
if successful, false
otherwise.
instance.update!(attributes)
: Assigns attributes and calls save!
. Raises an exception if saving fails (e.g., CQL::Errors::RecordInvalid
).
Model.update!(id, attributes)
(Class Method)This class method updates a record identified by its primary key with the given attributes. It typically loads the record, updates attributes, and calls save!
. Can raise DB::NoResultsError
if the ID is not found, or validation/save exceptions.
Model.update_by
, Model.update_all
)These methods allow updating multiple records at once without instantiating each one.
Model.update_by(conditions_hash, updates_hash)
: Updates all records matching conditions_hash
with the attributes in updates_hash
.
Model.update_all(updates_hash)
: Updates all records in the table with the attributes in updates_hash
. Use with extreme caution!
These methods typically execute a single SQL UPDATE
statement and do not instantiate records, run validations, or trigger callbacks.
Records can be deleted individually or in batches.
delete!
(Instance Method)Deletes the specific model instance from the database.
Runs before_destroy
and after_destroy
callbacks.
Returns true
if successful. Returns false
if a before_destroy
callback halts the operation.
Model.delete!(id : Pk)
(Class Method)Deletes the record with the specified primary key directly from the database.
This method typically does not run Active Record callbacks (before_destroy
, after_destroy
) as it usually issues a direct SQL DELETE
command.
It might raise an exception if the record doesn't exist or if there's a database error.
Model.delete_by!
, Model.delete_all
)These class methods delete multiple records based on conditions or all records from a table.
Model.delete_by!(attributes_hash)
: Deletes all records matching the attributes_hash
.
Model.delete_all
: Deletes all records from the model's table. Use with extreme caution!
These methods typically execute direct SQL DELETE
statements and do not instantiate records or run Active Record callbacks.
Always be careful with batch delete operations, especially delete_all
, as they can lead to irreversible data loss if not used correctly.
CQL provides various methods to retrieve records from the database. Many of these are covered in detail in the .
For building more complex queries (e.g., with joins, specific selections, grouping), refer to the .
This guide covers the primary CRUD operations available in CQL Active Record. For more advanced querying, refer to the , and for information on lifecycle events and data integrity, see the guides on and .
AcmeDB2 = CQL::Schema.build(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]) do
table :movies do
primary :id, Int64, auto_increment: true
text :title
end
table :screenplays do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :content
end
table :actors do
primary :id, Int64, auto_increment: true
text :name
end
table :directors do
primary :id, Int64, auto_increment: true
bigint :movie_id
text :name
end
table :movies_actors do
primary :id, Int64, auto_increment: true
bigint :movie_id
bigint :actor_id
end
end
MainDB = CQL::Schema.build(:main, adapter: CQL::Adapter::Postgres, uri: ENV["MAIN_DB_URL"]) do
# Define main schema tables
end
AnalyticsDB = CQL::Schema.build(:analytics, adapter: CQL::Adapter::Postgres, uri: ENV["ANALYTICS_DB_URL"]) do
# Define analytics schema tables
end
struct User
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :users
property id : Int64?
property name : String
property email : String
property active : Bool = false
# ... other properties and initializers ...
end
# Using save (returns true/false)
user1 = User.new(name: "John Doe", email: "john.doe@example.com")
if user1.save
puts "User '#{user1.name}' saved with ID: #{user1.id.not_nil!}"
else
puts "Failed to save user '#{user1.name}': #{user1.errors.full_messages.join(", ")}"
end
# Using save! (raises on failure)
try
user2 = User.new(name: "Jane Doe", email: "jane.doe@example.com")
user2.save!
puts "User '#{user2.name}' saved with ID: #{user2.id.not_nil!}"
rescue ex : CQL::Errors::RecordInvalid
puts "Validation error for '#{user2.name}': #{ex.record.errors.full_messages.join(", ")}"
rescue ex : Exception # Catch other potential save errors
puts "Could not save user '#{user2.name}': #{ex.message}"
end
# Using create! (raises on failure)
try
user3 = User.create!(name: "Alice Wonderland", email: "alice@example.com", active: true)
puts "User '#{user3.name}' created with ID: #{user3.id.not_nil!}"
# You can also pass a hash:
attrs = {name: "Bob The Builder", email: "bob@example.com"}
user4 = User.create!(attrs)
puts "User '#{user4.name}' created with ID: #{user4.id.not_nil!}"
rescue ex : CQL::Errors::RecordInvalid
puts "Failed to create user: #{ex.record.errors.full_messages.join(", ")}"
rescue ex : Exception
puts "Failed to create user: #{ex.message}"
end
# Using create (less common for direct use, as error handling requires checking instance.errors)
user5_attrs = {name: "Valid User", email: "valid@example.com"}
user5 = User.create(user5_attrs)
if user5 && user5.persisted?
puts "User '#{user5.name}' created."
elsif user5 # Instance returned but not persisted
puts "Failed to create user '#{user5.name}': #{user5.errors.full_messages.join(", ")}"
else # Create might return false directly
puts "User creation completely halted (e.g., by a before_create callback)."
end
# Attempts to find a user by email. If not found, creates with name and email.
user_charlie = User.find_or_create_by(email: "charlie@example.com", name: "Charlie Brown")
puts "User '#{user_charlie.name}' (ID: #{user_charlie.id.not_nil!}) is present."
# If called again with the same email, it will find the existing record.
# The name attribute here would be ignored if only email is used for finding.
user_charlie_again = User.find_or_create_by(email: "charlie@example.com", name: "Charles Brown (updated attempt)")
puts "Found user '#{user_charlie_again.name}' again (ID: #{user_charlie_again.id.not_nil!}). Name should be 'Charlie Brown' unless attributes were updated separately."
# It's common to pass all necessary attributes for creation:
user_diana = User.find_or_create_by(email: "diana@example.com", name: "Diana Prince", active: true)
puts "User '#{user_diana.name}' created or found."
all_users = User.all
maybe_user = User.find?(1_i64)
# user = User.find!(1_i64)
active_admin = User.find_by(active: true, role: "admin")
# specific_user = User.find_by!(email: "jane.doe@example.com")
all_active_users = User.find_all_by(active: true)
if user_to_update = User.find_by(email: "john.doe@example.com")
user_to_update.active = true
user_to_update.name = "Johnathan Doe"
# user_to_update.updated_at = Time.utc # Often handled by callbacks or DB
if user_to_update.save
puts "User '#{user_to_update.name}' updated."
else
puts "Failed to update '#{user_to_update.name}': #{user_to_update.errors.full_messages.join(", ")}"
end
end
# Using save! for updates (raises on failure)
if user_jane = User.find_by(email: "jane.doe@example.com")
user_jane.name = "Jane D. Updated"
begin
user_jane.save!
puts "User '#{user_jane.name}' updated successfully with save!"
rescue ex : CQL::Errors::RecordInvalid
puts "Failed to update '#{user_jane.name}': #{ex.record.errors.full_messages.join(", ")}"
rescue ex : Exception
puts "Failed to update '#{user_jane.name}': #{ex.message}"
end
end
if user_alice = User.find_by(email: "alice@example.com")
begin
# Using update!
user_alice.update!(active: false, name: "Alice Inactive")
puts "User '#{user_alice.name}' updated with instance update!"
# Using update (returns true/false)
if user_alice.update(name: "Alice Active Again")
puts "User '#{user_alice.name}' updated with instance update."
else
puts "Failed to update Alice with instance update: #{user_alice.errors.full_messages.join(", ")}"
end
rescue ex : Exception
puts "Exception during Alice's update!: #{ex.message}"
end
end
# Assuming user_bob was created earlier and user_bob.id is available
if user_bob_id = some_user_id_variable # e.g., user_bob.id.not_nil!
begin
User.update!(user_bob_id, active: true, name: "Robert The Great")
puts "User ID #{user_bob_id} updated via class update!"
# Can also take a hash for attributes
# User.update!(user_bob_id, {name: "Robert The Builder Again"})
rescue ex : DB::NoResultsError
puts "User ID #{user_bob_id} not found for update."
rescue ex : CQL::Errors::RecordInvalid
puts "Validation failed for User ID #{user_bob_id}: #{ex.record.errors.full_messages.join(", ")}"
rescue ex : Exception
puts "Failed to update User ID #{user_bob_id}: #{ex.message}"
end
end
# Example: Make all users with email ending in '@example.com' inactive.
# Assuming a direct SQL condition is needed or `update_by` supports patterns.
# The exact capabilities of `update_by` for conditions might vary; consult CQL specifics.
# For a simple equality condition:
User.update_by({role: "guest"}, {active: false, updated_at: Time.utc})
puts "Deactivated all guest users."
# Example: Update all users to have a default status (use carefully!)
# User.update_all({status: "pending_review", updated_at: Time.utc})
# puts "Set all users to pending_review status."
if user_to_delete = User.find_by(email: "charlie@example.com")
puts "Attempting to delete user: #{user_to_delete.name}"
if user_to_delete.delete!
puts "User '#{user_to_delete.name}' deleted successfully."
else
puts "Failed to delete user '#{user_to_delete.name}' (perhaps a before_destroy callback halted?)."
end
else
puts "User to delete not found."
end
if some_user = User.find_by(email: "user_to_delete_by_id@example.com")
user_id_to_delete = some_user.id.not_nil!
begin
User.delete!(user_id_to_delete)
puts "User ID #{user_id_to_delete} deleted via class delete! (callbacks likely skipped)."
rescue ex : Exception
puts "Failed to delete User ID #{user_id_to_delete} via class delete!: #{ex.message}"
end
end
# Delete all users marked as inactive
User.delete_by!(active: false)
puts "Deleted all inactive users (callbacks likely skipped)."
# Delete all records from the users table (USE WITH EXTREME CAUTION!)
# User.delete_all
# puts "Deleted all users from the table (callbacks likely skipped)."
Software design patterns are general, reusable solutions to commonly occurring problems within a given context in software design. In the realm of Object-Relational Mapping (ORM) and database interaction, several established patterns help structure how applications access and manipulate data.
This section explores some of the key data access and ORM patterns that can be implemented or are relevant when working with Crystal Query Language (CQL). Understanding these patterns can help you design more maintainable, scalable, and understandable data access layers in your Crystal applications.
We will delve into the following patterns:
Active Record: An object that wraps a row in a database table or view, encapsulates database access, and adds domain logic on that data.
Entity Framework (Conceptual Overview): While CQL is not a direct port of Microsoft's Entity Framework, this section will discuss some conceptual similarities or how related ideas (like a rich object context and LINQ-style querying) might apply or inspire usage with CQL.
Repository: Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects.
Each pattern offers different trade-offs in terms of simplicity, flexibility, testability, and separation of concerns. Explore the specific guides to understand how they can be applied with CQL.
In the context of CQL (Crystal Query Language), Entity Framework (EF) serves as a useful comparison to help developers understand how CQL and its features (like migrations, schema management, and object-relational mapping) work. Just as EF simplifies database interactions in .NET applications, CQL does the same for Crystal applications. Let’s break down the key concepts and approaches of EF and see how they align with CQL’s functionalities.
In Entity Framework, developers have three approaches to database design: Database-First, Code-First, and Model-First. CQL shares some similarities, especially with the Code-First and Database-First approaches, but with Crystal-specific tooling.
CQL primarily uses a Schema-First approach, where you define your database schema using Crystal code and CQL builds and manages the database based on this schema. This is similar to EF’s Code-First approach, where the developer defines the entity classes and EF generates the database schema.
Example in CQL:
schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
table :users do
primary :id, Int32
column :name, String
column :age, Int32
end
end
schema.init
This code defines the users
table and its columns (id
, name
, and age
), which CQL will use to generate the corresponding database structure.
DbContext
and DbSet
)In EF, a DbContext
is used to manage the entities (mapped to database tables), and each DbSet
represents a collection of entities (mapped to individual tables). Similarly, in CQL:
CQL::Schema
manages the structure of your database.
Tables are defined within the schema using table
blocks.
Example:
schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
table :products do
primary :id, Int32
column :name, String
column :price, Float64
end
end
This is similar to defining DbSet<Product>
in EF, which represents the products
table.
In CQL, migrations are a key feature, similar to EF’s migrations, for managing database schema changes over time. Migrations allow you to evolve your schema, apply changes, roll them back, and maintain a history of modifications.
Example migration in CQL:
class CreateUsersTable < CQL::Migration
self.version = 1_i64
def up
schema.alter :users do
add_column :name, String
add_column :age, Int32
end
end
def down
schema.alter :users do
drop_column :name
drop_column :age
end
end
end
In EF, migrations are handled using commands like Add-Migration
and Update-Database
. In CQL, migrations are applied and managed using the CQL::Migrator
class, which provides methods for applying, rolling back, and listing migrations.
Applying migrations in CQL:
migrator = CQL::Migrator.new(schema)
migrator.up # Apply all pending migrations
This is similar to EF’s Update-Database
command, which applies migrations to the database.
In CQL, database tables and columns are defined as part of the schema, and relationships like foreign keys can be established. This is comparable to how entities and relationships between them are managed in EF.
Example in CQL (adding a foreign key):
schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
table :orders do
primary :id, Int32
bigint :user_id
text :order_date
foreign_key :user_id, :users, :id, on_delete: "CASCADE"
end
end
This defines an orders
table with a foreign key relationship to the users
table, similar to how EF handles one-to-many or many-to-many relationships.
CQL’s migration life cycle aligns closely with EF’s migrations system. In both cases, you:
Define migrations to introduce schema changes.
Apply migrations to update the database schema.
Rollback or redo migrations if needed to manage schema changes.
migrator.up # Apply pending migrations
migrator.down # Rollback the last migration
migrator.redo # Rollback and reapply the last migration
In EF, you would use commands like Update-Database
, Remove-Migration
, and Add-Migration
to manage these operations.
In EF, developers use LINQ to query entities and interact with the database. CQL also allows querying the database but through SQL-like syntax, aligning with Crystal’s programming style.
Example query in CQL:
AcmeDB
.query
.from(:products)
.select(:name, :price)
.where(name: "Laptop")
.all(Product)
This retrieves all products where the name is "Laptop", similar to how LINQ queries work in EF:
var laptops = dbContext.Products.Where(p => p.Name == "Laptop").ToList();
Both frameworks provide abstractions that avoid writing raw SQL, but CQL maintains a more SQL-like approach in its query building.
Productivity: Like EF, CQL reduces the need for writing raw SQL by allowing developers to work with objects (schemas, tables, columns).
Schema Management: CQL migrations simplify managing database changes, ensuring your schema evolves without breaking changes.
Consistency: CQL’s Schema-First approach ensures the database schema is in sync with your application’s Crystal code, similar to EF’s Code-First approach.
CQL provides similar ORM-like features for Crystal that Entity Framework does for .NET. Whether using migrations, schema definitions, or querying data, both CQL and EF streamline database interactions, making it easier to manage complex applications. By understanding the core parallels between these two systems, Crystal developers can better leverage CQL’s powerful schema management and migration tools to build scalable and maintainable database-driven applications.
Welcome to CQL Active Record! This guide will help you set up CQL in your Crystal project, connect to a database, define your first model, and perform basic CRUD, and first query. Use idiomatic Crystal and CQL, and provide clear, step-by-step examples.
Add CQL to your shard.yml
:
dependencies:
cql:
github: your-org/cql
version: ~> 1.0
Then install dependencies:
shards install
Configure your database connection. For example, to use PostgreSQL:
require "cql"
require "cql/pg"
# Define a database context
AcmeDB = CQL::DBContext.new(
CQL::PG::Driver.new(
host: "localhost",
user: "postgres",
password: "password",
database: "acme_db"
)
)
Create a model that maps to a table:
struct User
include CQL::ActiveRecord::Model(Int32)
db_context AcmeDB, :users
property id : Int32?
property name : String
property email : String
property active : Bool = false
property created_at : Time?
property updated_at : Time?
end
Create your tables using migrations (optional but recommended):
class CreateUsers < CQL::Migration
self.version = 1_i64
def up
schema.create :users do
primary_key :id, :serial
column :name, :string
column :email, :string
column :active, :bool, default: false
column :created_at, :timestamp
column :updated_at, :timestamp
end
end
def down
schema.drop :users
end
end
Run your migrations using your preferred migration runner.
user = User.new(name: "Alice", email: "alice@example.com")
user.save
user = User.query.where(name: "Alice").first(User)
puts user.try(&.email)
user = User.query.where(name: "Alice").first(User)
if user
user.active = true
user.save
end
user = User.query.where(name: "Alice").first(User)
user.try(&.delete)
# Find all active users
active_users = User.query.where(active: true).all(User)
active_users.each { |u| puts u.name }
This guide explores how to leverage database transactions effectively within your Crystal applications using CQL's Active Record pattern. We will use a practical banking domain example throughout to illustrate key concepts and best practices.
This documentation covers the functionality provided by the @transactions.md
and @transactional.cr
modules within the library.
In database systems, a transaction is a single unit of work. This unit comprises one or more operations that are treated as an indivisible sequence. The core principle is that either all operations within the transaction complete successfully (commit) or none of them do (rollback).
Imagine a simple task like transferring money between two bank accounts. This isn't just one database operation; it typically involves:
Debiting the sender's account.
Crediting the recipient's account.
If the debit succeeds but the credit fails (e.g., due to a network error or a constraint violation), the system would be in an inconsistent state – money is gone from one account but didn't appear in the other. Transactions prevent this by ensuring that both steps must succeed together. If the credit fails, the debit is automatically undone.
In application development, especially when dealing with complex business logic and multiple related data changes, transactions are crucial for:
Maintaining Data Consistency: Ensuring that your database adheres to all predefined rules and constraints.
Preventing Partial Updates: Avoiding scenarios where only a portion of a multi-step operation completes, leaving data in an invalid state.
Handling Concurrency: Providing a mechanism to manage simultaneous access and modifications to data by multiple users or processes (though concurrency requires careful consideration of isolation levels and potential deadlocks).
Simplifying Error Recovery: If an error occurs within a transaction, you know the database will revert to its state before the transaction began, making error handling and recovery more predictable.
Transactions provide the crucial ACID properties:
Atomicity: The transaction is a single unit; either it completes entirely or has no effect.
Consistency: A transaction brings the database from one valid state to another.
Isolation: Concurrent transactions do not interfere with each other. Each sees the database as if it were running alone.
Durability: Once a transaction is committed, the changes are permanent and survive system failures.
CQL's Active Record implementation simplifies transaction management in Crystal. By including the CQL::ActiveRecord::Transactional
module in your model, you gain access to the transaction
class method. This method provides a block-based interface for executing a series of database operations within a single transaction.
To demonstrate transactions, we will use a simplified banking application.
The core operation is transferring money between accounts. This operation must be atomic – the debit from one account and the credit to another must happen together. We also need to track these operations for auditing purposes.
BankAccount: Stores account details and balance.
Transaction: Records deposits, withdrawals, and transfers.
AuditLog: Provides a detailed log of system activities.
(Detailed schema and model definitions are omitted for brevity, but assume standard columns like id
, balance
, amount
, type
, created_at
, etc., as introduced in the schema section of previous versions).
The core logic for banking operations like withdrawals, deposits, and transfers involves updating one or more BankAccount
records, creating a Transaction
record, and often creating an AuditLog
record. These steps must be performed atomically. This is where the BankAccount.transaction
block becomes essential.
BankAccount.transaction
BlockAny database operations performed using CQL Active Record methods inside a BankAccount.transaction do ... end
block are treated as a single unit by the database.
When the block is entered, a database transaction is started (BEGIN
).
When the block completes without raising an exception, the transaction is committed (COMMIT
), making all changes permanent.
If any exception is raised within the block, the transaction is automatically rolled back (ROLLBACK
), discarding all changes made during the transaction.
You can also explicitly call tx.rollback
on the yielded transaction object tx
to manually roll back.
This ensures that if any step of a multi-operation process fails, the entire set of operations is undone, maintaining data integrity.
Here are the core transaction blocks for our banking operations, demonstrating the use of CQL features within the atomic unit:
This transaction debits a BankAccount
and records the event.
Why a Transaction is Useful Here: While a withdrawal is simpler than a transfer, using a transaction ensures that the account balance update and the recording of the transaction/audit log happen together. If the record-keeping fails, the balance change is undone.
This transaction credits a BankAccount
and records the event.
Why a Transaction is Useful Here: Similar to withdrawal, it guarantees that the balance update and the logging/recording of the deposit happen together.
This is the prime example where transactions are critical. Money must be debited from one account and credited to another and records created, all or nothing.
Why a Transaction is Crucial Here: This is the quintessential transaction use case. The debit and credit must happen together. If the debit succeeds but the credit fails (or vice versa, or if the transaction/audit log creation fails), the transaction ensures that all changes are undone, preventing money from being lost or duplicated. It guarantees Atomicity.
As shown in the conceptual examples, any unhandled exception raised within the BankAccount.transaction do |tx| ... end
block will automatically trigger a database rollback. This is a key feature that ensures atomicity.
You can also explicitly roll back the transaction using the transaction object yielded to the block:
When tx.rollback
is called, the database driver is instructed to perform a rollback. Any subsequent operations within the block before it exits will also be part of the rolled-back transaction.
Use CQL's query builder (.query
, .where
, .order
, .all
, .exists?
) to inspect the records created by successful transactions:
While CQL transactions automatically commit if the block finishes without an unhandled exception and automatically roll back if an exception occurs, you can also explicitly control the outcome using tx.commit
and tx.rollback
on the yielded transaction object.
Explicit Rollback:
You can manually trigger a rollback at any point within the transaction block. This is useful if business logic dictates that a transaction should not proceed, even if no technical error (exception) has occurred.
Explicit Commit:
Similarly, you can explicitly commit a transaction before the end of the block. This can be useful in more complex scenarios, though it's less common than explicit rollback or relying on the implicit commit.
Important Note:
As stated in the Crystal DB documentation: After commit
or rollback
are used, the transaction is no longer usable. The connection is still open but any statement will be performed outside the context of the terminated transaction. This means you should typically not perform further database operations relying on that specific tx
object after calling tx.commit
or tx.rollback
.
CQL supports nested transactions using database savepoints. This is useful for sub-operations within a larger transaction that might need to be rolled back independently without affecting the outer transaction.
To create a nested transaction, pass an existing transaction object (e.g., outer_tx
) to the Model.transaction
method: Model.transaction(outer_tx) do |inner_tx| ... end
. This creates a SAVEPOINT.
Here are common scenarios:
Both nested and outer operations are committed.
inner_tx.rollback
), Outer CommitOnly inner operations roll back. Outer operations commit.
After inner_tx.rollback
, the inner_tx
object is no longer usable for DB operations in that context.
raise DB::Rollback
), Outer CommitDB::Rollback
in the inner transaction rolls back only inner operations. The exception is handled internally, allowing the outer transaction to commit.
Exception
), Entire Transaction Rolls BackA standard exception in the inner block, if not caught and handled within that inner block, rolls back both inner and outer transactions.
If the inner transaction completes but the outer transaction subsequently rolls back, all changes (inner and outer) are discarded.
Key Summary Points:
Savepoints: Nested transactions (Model.transaction(outer_tx)
) use database SAVEPOINT
s.
inner_tx.rollback
: Rolls back only the inner transaction to its savepoint. The outer transaction can continue and commit.
raise DB::Rollback
in Inner: Same effect as inner_tx.rollback
. The DB::Rollback
exception is handled by the inner transaction logic and doesn't cause the outer transaction to fail.
Other Exceptions in Inner: If not caught within the inner block, will roll back the inner transaction and propagate to roll back the outer transaction.
Outer Rollback: If the outer transaction rolls back, all work (including successful inner transactions) is undone.
Commit: Inner transaction changes are permanent only if the outermost transaction commits.
Database Driver Dependency: Behavior relies on the database and driver supporting savepoints.
Use nested transactions judiciously for sub-units needing independent rollback within a larger atomic operation, as they add complexity.
Follow these best practices when working with transactions in CQL:
Keep transactions short and focused: Minimize the amount of work inside a transaction block. Long-running transactions hold locks longer, increasing contention and deadlocks.
Validate data before entering transactions: Perform necessary validation before starting the transaction to avoid unnecessary rollbacks.
Encapsulate logic: Wrap complex transactional logic in dedicated methods or Service Objects for organization and testability.
Handle exceptions: Catch exceptions outside the block for better logging and error handling, while relying on the automatic rollback inside.
Ensure Consistent Resource Access Order: Access multiple records within a transaction in a consistent order (e.g., by primary key ID) to reduce deadlocks.
Consider Isolation Levels: Understand and potentially configure isolation levels for advanced concurrency.
Avoid Nested Transactions: Rely on the single outer transaction.
Deadlocks: Caused by transactions waiting for each other's locks. Consistent resource ordering helps.
Silent Transaction Failures: Ensure errors inside transactions are handled and reported properly outside the block.
Connection Issues: Implement retry logic for transient database connection problems.
Misunderstanding Rollback: Rollback affects all database operations within the transaction block.
By diligently applying these principles and patterns, you can effectively use CQL's transaction capabilities to build robust and reliable applications that maintain data integrity even in the face of errors or concurrent access.
# Include this module in your model
struct MyModel
include CQL::ActiveRecord::Model(Int32)
include CQL::ActiveRecord::Transactional # <--- Gives access to .transaction
db_context MY_DB, :my_models
# ... properties
end
# Use the transaction block
MyModel.transaction do |tx|
# All database operations within this block form a single transaction
# If an unhandled exception occurs, the transaction is automatically rolled back
# You can also use tx.rollback to manually roll back
end
# Conceptual method demonstrating the core transaction logic for withdrawal
def perform_withdrawal_transaction(account : BankAccount, amount : Float64)
# Assumes validation (amount > 0, sufficient funds) happened BEFORE this block
BankAccount.transaction do |tx|
# 1. Update the account balance using CQL Active Record save!
account.balance -= amount
account.updated_at = Time.utc
account.save! # Persists the balance change within the transaction
# 2. Create a transaction record using CQL Active Record create!
Transaction.create!( # Creates a new record within the transaction
amount: amount,
transaction_type: "withdrawal",
from_account_id: account.id,
status: "completed",
created_at: Time.utc
)
# 3. Create an audit log entry using CQL Active Record create!
AuditLog.create!( # Creates a new record within the transaction
action: "withdrawal",
entity_type: "bank_account",
entity_id: account.id.not_nil!,
data: { account: account.account_number, amount: amount }.to_json,
created_at: Time.utc
)
# If any of the above .save! or .create! calls fail (e.g., DB error),
# or if an exception is raised, the transaction will roll back automatically.
end
end
# Conceptual method demonstrating the core transaction logic for deposit
def perform_deposit_transaction(account : BankAccount, amount : Float64)
# Assumes validation (amount > 0) happened BEFORE this block
BankAccount.transaction do |tx|
# 1. Update the account balance
account.balance += amount
account.updated_at = Time.utc
account.save! # Persists the balance change within the transaction
# 2. Create a transaction record
Transaction.create!( # Creates a new record within the transaction
amount: amount,
transaction_type: "deposit",
to_account_id: account.id,
status: "completed",
created_at: Time.utc
)
# 3. Create an audit log entry
AuditLog.create!( # Creates a new record within the transaction
action: "deposit",
entity_type: "bank_account",
entity_id: account.id.not_nil!,
data: { account: account.account_number, amount: amount }.to_json,
created_at: Time.utc
)
end
end
# Conceptual method demonstrating the core transaction logic for transfer
def perform_transfer_transaction(from_account : BankAccount, to_account : BankAccount, amount : Float64)
# Assumes validation (amount > 0, different accounts, sufficient funds)
# happened BEFORE this block.
# Also assumes accounts are sorted by ID outside for deadlock prevention.
BankAccount.transaction do |tx|
# Operations within the transaction - ENSURING ATOMICITY
# 1. Reload accounts to get freshest data under current isolation level
# This is crucial for handling concurrent access. Use CQL Active Record reload.
from_account.reload
to_account.reload
# Re-validate insufficient funds after reloading
raise "Insufficient funds after reload" if from_account.balance < amount
# 2. Debit the sender's account using CQL Active Record save!
from_account.balance -= amount
from_account.updated_at = Time.utc
from_account.save! # Persists the debit within the transaction
# 3. Credit the recipient's account using CQL Active Record save!
to_account.balance += amount
to_account.updated_at = Time.utc
to_account.save! # Persists the credit within the transaction
# 4. Create a transaction record using CQL Active Record create!
Transaction.create!( # Creates a new record within the transaction
amount: amount,
transaction_type: "transfer",
from_account_id: from_account.id,
to_account_id: to_account.id,
status: "completed",
created_at: Time.utc
)
# 5. Create an audit log entry using CQL Active Record create!
AuditLog.create!( # Creates a new record within the transaction
action: "money_transfer",
entity_type: "transfer",
entity_id: nil, # No single entity ID for the transfer action
data: { from: from_account.account_number, to: to_account.account_number, amount: amount }.to_json,
created_at: Time.utc
)
# If any of these steps (reloading, saving, creating records) fails,
# the entire transaction is rolled back, guaranteeing that no partial
# changes are saved to the database.
end
end
BankAccount.transaction do |tx|
# Perform some operations...
if some_business_condition_is_not_met
puts "Condition not met, rolling back!"
tx.rollback # Explicitly roll back all changes made so far in this block
# Note: Raising an exception immediately after tx.rollback is common
# to stop execution and indicate failure.
raise "Manual rollback triggered"
end
# If no rollback or exception occurs, the transaction is committed here
end
# Get all transfers from Alice's account
alice_transfer_history = Transaction.query
.where(from_account_id: alice_account.id)
.order(created_at: :desc)
.all(Transaction) # Fetch all matching records as Transaction objects
puts "\n--- Alice's Transfer History ---"
alice_transfer_history.each { |tx| puts "- $#{tx.amount} to Account ID: #{tx.to_account_id} (#{tx.status})" }
# Get all audit logs related to Bob's account
bob_account_audit_logs = AuditLog.query
.where(entity_type: "bank_account", entity_id: bob_account.id)
.order(created_at: :asc)
.all(AuditLog) # Fetch all matching records as AuditLog objects
puts "\n--- Bob's Account Audit Logs ---"
bob_account_audit_logs.each { |log| puts "- #{log.action}: #{log.data}" }
BankAccount.transaction do |tx|
from_account = BankAccount.find_by(account_number: "ACC_SENDER")
to_account = BankAccount.find_by(account_number: "ACC_RECEIVER")
transfer_amount = 50.0
unless from_account && to_account
puts "Error: One or both accounts not found."
tx.rollback # Explicitly roll back
# Consider raising an exception or returning early after rollback
# to prevent further operations within this block.
raise "Account lookup failed, transaction rolled back."
end
if from_account.balance < transfer_amount
puts "Insufficient funds. Rolling back transaction."
tx.rollback # Explicitly roll back
raise "Insufficient funds, transaction rolled back."
end
# Proceed with operations if checks pass
from_account.balance -= transfer_amount
from_account.save!
to_account.balance += transfer_amount
to_account.save!
puts "Transfer appears successful before explicit decision."
# If we reach here, an implicit commit would happen at the end of the block.
# However, we could also make an explicit decision.
end
BankAccount.transaction do |tx|
account = BankAccount.create!(account_number: "ACC789", balance: 200.0)
puts "Account ACC789 created with balance #{account.balance}"
# Perform some critical updates
account.balance += 100.0
account.save!
puts "Balance updated to #{account.balance}"
# At this point, we decide to commit the changes immediately.
puts "Explicitly committing transaction."
tx.commit
# Further operations here would be outside the just-committed transaction.
# For example, trying to use `account.save!` again would likely start a new implicit transaction
# or operate outside any transaction if auto-commit is on for the connection.
# It is generally best practice to exit the block or not perform further DB
# operations relying on this specific transaction after an explicit commit/rollback.
end
BankAccount.transaction do |outer_tx|
account = BankAccount.create!(account_number: "ACC001", balance: 100.0)
# Outer operation
BankAccount.transaction(outer_tx) do |inner_tx|
account.balance += 50.0
account.save!
# Inner operation: changes are staged
end
# Outer transaction commits: all changes (outer + inner) are now permanent.
end
# Expected: ACC001 has balance 150.0
BankAccount.transaction do |outer_tx|
account = BankAccount.create!(account_number: "ACC002", balance: 100.0)
# Outer operation
BankAccount.transaction(outer_tx) do |inner_tx|
account.balance += 50.0
account.save! # Staged change
inner_tx.rollback # Rolls back the +50.0
end
# account.reload # In-memory `account` might be stale; DB reflects rollback.
# Outer transaction commits: only initial creation is permanent.
end
# Expected: ACC002 has balance 100.0
BankAccount.transaction do |outer_tx|
account = BankAccount.create!(account_number: "ACC003", balance: 100.0)
# Outer operation
begin
BankAccount.transaction(outer_tx) do |inner_tx|
account.balance += 50.0
account.save! # Staged change
raise DB::Rollback.new # Rolls back +50.0; outer transaction continues
end
rescue ex # Catches other exceptions, not DB::Rollback from inner_tx.transaction
puts "This line should not be reached by DB::Rollback: #{ex}"
end
# account.reload # DB reflects inner rollback.
# Outer transaction commits.
end
# Expected: ACC003 has balance 100.0
begin
BankAccount.transaction do |outer_tx|
account = BankAccount.create!(account_number: "ACC004", balance: 100.0)
# Outer operation
BankAccount.transaction(outer_tx) do |inner_tx|
account.balance += 50.0
account.save! # Staged change
raise "Inner operation failed!" # This exception will propagate
end
# This part is not reached
end
rescue StandardError => e
puts "Transaction failed: #{e.message}" # Logs "Inner operation failed!"
end
# Expected: ACC004 does not exist or transaction rolled back.
begin
BankAccount.transaction do |outer_tx|
account = BankAccount.create!(account_number: "ACC005", balance: 100.0)
# Outer operation
BankAccount.transaction(outer_tx) do |inner_tx|
account.balance += 50.0
account.save!
# Inner operations successful relative to its savepoint
end
raise "Outer operation failed post-inner success!" # Causes full rollback
end
rescue StandardError => e
puts "Transaction failed: #{e.message}" # Logs "Outer operation failed post-inner success!"
end
# Expected: ACC005 does not exist or transaction rolled back.
In this guide, we'll cover the HasOne
relationship using CQL's Active Record syntax. Like in the previous BelongsTo
guide, we'll start with an Entity-Relationship Diagram (ERD) to visually represent how the HasOne
relationship works and build on the structure we already introduced with the BelongsTo
relationship.
HasOne
Relationship?The HasOne
relationship indicates that one entity (a record) is related to exactly one other entity. For example, a User can have one Profile associated with it. This relationship is a one-to-one mapping between two entities.
Let's say we have a system where:
A User can have one Profile.
A Profile belongs to one User.
We will represent this one-to-one relationship using CQL's HasOne
and BelongsTo
associations.
We'll define the users
and profiles
tables in the schema using CQL.
AcmeDB = CQL::Schema.define(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :users do
primary
text :name
text :email
end
table :profiles do
primary
bigint :user_id, index: true
text :bio
text :avatar_url
end
end
users table: Stores user details like name
and email
.
profiles table: Stores profile details like bio
and avatar_url
. It has a user_id
foreign key referencing the users
table.
Let's define the User
and Profile
models in CQL, establishing the HasOne
and BelongsTo
relationships.
struct User
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :users
property id : Int64?
property name : String
property email : String
# Initializing a new user with name and email
def initialize(@name : String, @email : String)
end
# Association: A User has one Profile
has_one :profile, Profile
end
The has_one :profile, Profile
association in the User
model indicates that each user has one profile. The foreign key (e.g., user_id
) is expected on the profiles
table.
struct Profile
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :profiles
property id : Int64?
property user_id : Int64?
property bio : String
property avatar_url : String
# Initializing a profile. User can be associated later.
def initialize(@bio : String, @avatar_url : String, @user_id : Int64? = nil)
end
# Association: A Profile belongs to one User
belongs_to :user, User, foreign_key: :user_id
end
The belongs_to :user, User, foreign_key: :user_id
association in the Profile
model links each profile to a user.
Now that we have defined the User
and Profile
models with has_one
and belongs_to
relationships, let's see how to create and query records.
Option 1: Create User, then use create_association
for Profile
# 1. Create and save the User
user = User.new(name: "John Doe", email: "john@example.com")
user.save!
# 2. Create and associate the Profile using the helper method
# This creates a Profile, sets its user_id to user.id, and saves the Profile.
profile = user.create_profile(bio: "Developer at Acme", avatar_url: "avatar_url.jpg")
puts "User #{user.name} created with ID: #{user.id.not_nil!}"
puts "Profile for user created with ID: #{profile.id.not_nil!}, Bio: #{profile.bio}, UserID: #{profile.user_id}"
Option 2: Create User, then build and save Profile manually
user2 = User.new(name: "Jane Roe", email: "jane@example.com")
user2.save!
# Build the profile (in memory, foreign key is set)
new_profile = user2.build_profile(bio: "Designer at Innovations", avatar_url: "jane_avatar.png")
# new_profile.user_id is now user2.id
# You must save the profile separately if you use build_profile
new_profile.save!
puts "User #{user2.name} created with ID: #{user2.id.not_nil!}"
puts "Built and saved Profile ID: #{new_profile.id.not_nil!}, UserID: #{new_profile.user_id}"
Option 3: Manual creation (less common with helpers available)
user3 = User.new(name: "Manual Mike", email: "mike@example.com")
user3.save!
profile_manual = Profile.new(bio: "Manual bio", avatar_url: "manual.jpg")
profile_manual.user_id = user3.id # Manually set foreign key
# or profile_manual.user = user3 # Use belongs_to setter if preferred before save
profile_manual.save!
puts "User #{user3.name} (ID: #{user3.id.not_nil!}) and Profile (ID: #{profile_manual.id.not_nil!}) created manually."
Once a user and their profile have been created, you can retrieve the profile using the has_one
association.
# Fetch the user
user = User.find(1)
# Fetch the associated profile
profile = user.profile
puts profile.bio # Outputs: "Developer at Acme"
Here, user.profile
fetches the profile associated with the user.
Similarly, you can retrieve the associated user from the profile.
# Fetch the profile
profile = Profile.find(1)
# Fetch the associated user
user = profile.user
puts user.name # Outputs: "John Doe"
In this example, profile.user
fetches the User
associated with that Profile
.
You can also delete the associated profile.
# Fetch the user
user_for_delete_test = User.find_by!(email: "john@example.com") # Assuming John exists
# Option 1: Using the association getter and then instance delete
if existing_profile = user_for_delete_test.profile
existing_profile.delete!
puts "Profile for #{user_for_delete_test.name} deleted via instance delete."
end
# Re-create for next example
user_for_delete_test.create_profile(bio: "Temporary bio", avatar_url: "temp.jpg")
# Option 2: Using the has_one delete helper (if available and user.profile exists)
if user_for_delete_test.delete_profile # This method returns true if successful
puts "Profile for #{user_for_delete_test.name} deleted via user.delete_profile helper."
else
puts "Could not delete profile for #{user_for_delete_test.name} via helper, or no profile existed."
end
Similarly, deleting the user will not automatically delete the associated profile unless cascade rules are explicitly set in the database or handled by before_destroy
callbacks on the User model.
In this guide, we explored the has_one
relationship in CQL. We:
Define the User
and Profile
tables in the schema.
Created corresponding models, specifying the has_one
relationship in the User
model and the belongs_to
relationship in the Profile
model.
Demonstrated how to create, query, update, and delete records using the has_one
and belongs_to
associations.
In the next guide, we'll extend the ERD and cover the has_many
relationship, which is commonly used when one entity is associated with multiple records (e.g., a post having many comments).
Feel free to experiment with the has_one
relationship by adding more fields to your models, setting up validations, or extending your schema with more complex relationships.
CQL Active Record models are Crystal struct
s that map directly to database tables. Each model encapsulates the table's columns as properties, provides type-safe access to data, and includes methods for persistence, querying, and associations.
To define a model:
Key points:
Use struct
for models (Crystal convention for value types).
include CQL::ActiveRecord::Model(PkType)
mixes in all Active Record features. The type parameter specifies the primary key type (e.g., Int32
, Int64
, UUID
).
db_context AcmeDB, :users
links the model to a specific table in a database context.
Use property
for each column. Nullable types (e.g., Int32?
) are used for columns that may be NULL
or auto-generated.
The primary key type is specified in the Model
inclusion.
By convention, the primary key property is named id
.
If the primary key is auto-generated, make it nullable (id : Int32?
).
Use the generated getter and setter methods:
Use the attributes
method to get a hash of all attribute values:
Set multiple attributes at once using a hash or keyword arguments:
Note: These methods only update the instance in memory. Call save
to persist changes.
Get all attribute names as symbols:
Use property
for all columns you want to map.
Use nullable types for columns that may be NULL
or auto-generated.
Use mass assignment carefully, especially with user input.
Use direct property access for individual attributes; use attributes
for bulk operations.
For more on querying and persistence, see the other guides in this directory.
# src/models/user.cr
require "cql"
struct User
# Include Active Record functionality with the primary key type
include CQL::ActiveRecord::Model(Int32)
# Map to the 'users' table in the AcmeDB context
db_context AcmeDB, :users
# Define properties for each column
property id : Int32? # Primary key, nullable if auto-generated
property name : String
property email : String
property active : Bool = false # Default value
property created_at : Time?
property updated_at : Time?
end
struct Product
include CQL::ActiveRecord::Model(UUID)
db_context StoreDB, :products
property id : UUID?
property name : String
property price : Float64
end
user = User.new(name: "Alice", email: "alice@example.com")
puts user.name # => "Alice"
user.name = "Bob"
attrs = user.attributes
# => {:id => 1, :name => "Alice", :email => "alice@example.com", ...}
puts attrs[:name]
user = User.new(name: "Placeholder", email: "placeholder@example.com")
# Using a hash
user.attributes = {name: "Jane", email: "jane@example.com", active: true}
# Using keyword arguments
user.attributes(name: "John", email: "john@example.com")
User.attribute_names # => [:id, :name, :email, :active, :created_at, :updated_at]
struct Article
include CQL::ActiveRecord::Model(Int64)
db_context BlogDB, :articles
property id : Int64?
property title : String
property body : String
property published : Bool = false
property author_id : Int32
property created_at : Time?
property updated_at : Time?
end
Welcome to the CQL Guides section! Here you'll find practical, focused documentation to help you use CQL (Crystal Query Language) effectively in your Crystal projects.
Active Record with CQL: Learn how to define models, perform CRUD operations, build queries, manage relationships, and ensure data integrity using CQL's Active Record pattern.
Querying: Explore the powerful query interface, including chainable queries, aggregations, and advanced filtering.
Migrations: Manage your database schema changes over time with CQL's migration system.
Relationships: Understand and implement associations like belongs_to
, has_one
, has_many
, and many_to_many
.
Other Topics: Find guides on pagination, scopes, callbacks, validations, and more.
Each guide is organized to be self-contained and practical, with code examples and explanations tailored for Crystal developers.
Start with Active Record with CQL for a comprehensive overview and links to all sub-guides.
Use the sidebar or the links within each guide to navigate to specific topics.
Refer to the Getting Started guide if you're new to CQL.
Whether you're building your first Crystal app with CQL or looking to master advanced features, these guides are your go-to resource.
The CQL::Update
class in the CQL (Crystal Query Language) module is designed to represent and execute SQL UPDATE
statements in a clean and structured manner. This guide will walk you through using the class to update records in a database, providing real-world examples and detailed explanations for each method.
Update records in a database with a simple and readable syntax.
Set column values dynamically using hashes or keyword arguments.
Filter records with flexible WHERE
conditions.
Return updated columns after executing the query.
Chainable methods for building complex queries effortlessly.
Let’s start with a simple example of updating a user’s name and age in the users
table.
update = CQL::Update.new(schema)
.table(:users)
.set(name: "John", age: 30)
.where { |w| w.id == 1 }
.commit
This example updates the user with id = 1
to have the name "John" and age 30.
Below is a detailed breakdown of the key methods in the CQL::Update
class and how to use them.
table(table : Symbol)
Purpose: Specifies the table to update.
Parameters: table
— A symbol representing the table name.
Returns: Update
object (for chaining).
Real-World Example: Setting the Target Table
update.table(:users)
This sets the users
table as the target for the update operation.
set(setters : Hash(Symbol, DB::Any))
Purpose: Specifies the column values to update using a hash.
Parameters: setters
— A hash where keys are column names and values are the new values for those columns.
Returns: Update
object (for chaining).
Real-World Example: Updating Multiple Columns
update
.table(:users)
.set(name: "John", age: 30)
This sets the name
and age
columns to new values for the target record(s).
set(**fields)
Purpose: Specifies the column values to update using keyword arguments.
Parameters: fields
— Column-value pairs as keyword arguments.
Returns: Update
object (for chaining).
Real-World Example: Using Keyword Arguments
update
.table(:users)
.set(name: "Alice", active: true)
This sets the name
to "Alice" and active
to true
.
where(**fields)
Purpose: Adds a WHERE
clause to filter the records to be updated.
Parameters: fields
— A hash where keys are column names and values are the conditions to match.
Returns: Update
object (for chaining).
Real-World Example: Filtering by a Condition
update
.table(:users)
.set(name: "John", age: 30)
.where(id: 1)
This adds a condition to only update the user where id = 1
.
where(&block)
Purpose: Adds a WHERE
clause using a block for more complex conditions.
Parameters: Block that db_contexts the condition using a filter builder.
Returns: Update
object (for chaining).
Real-World Example: Using a Block for Conditions
update
.table(:users)
.set(name: "John")
.where { |w| w.id == 1 && w.active == true }
This example updates the user where both id = 1
and active = true
.
commit
Purpose: Executes the UPDATE
query and commits the changes to the database.
Returns: A DB::Result
object, which represents the result of the query execution.
Real-World Example: Committing the Update
update = CQL::Update.new(schema)
.table(:users)
.set(name: "John", age: 30)
.where { |w| w.id == 1 }
.commit
This commits the changes to the users
table, updating the user with id = 1
.
back(*columns : Symbol)
Purpose: Specifies the columns to return after the update.
Parameters: columns
— An array of symbols representing the columns to return.
Returns: Update
object (for chaining).
Real-World Example: Returning Updated Columns
update
.table(:users)
.set(name: "John", age: 30)
.where(id: 1)
.back(:name, :age)
.commit
This will return the updated name
and age
columns after the update.
to_sql(gen = @schema.gen)
Purpose: Generates the SQL query and the parameters required for the UPDATE
statement.
Parameters: gen
— The generator used for SQL generation (default: schema generator).
Returns: A tuple containing the SQL query string and the parameters.
Real-World Example: Generating SQL for an Update
update = CQL::Update.new(schema)
.table(:users)
.set(name: "John", age: 30)
.where(id: 1)
sql, params = update.to_sql
puts sql # "UPDATE users SET name = $1, age = $2 WHERE id = $3"
puts params # ["John", 30, 1]
This generates the raw SQL query and its associated parameters without executing it.
Let’s combine multiple methods to handle a more advanced use case. Suppose you want to update a user's data, but only if they are active, and you want to return their updated email address afterward:
update = CQL::Update.new(schema)
result = update
.table(:users)
.set(name: "Charlie", email: "charlie@example.com")
.where { |w| w.id == 1 && w.active == true }
.back(:email)
.commit
puts result # This will return the updated email address of the user.
In this query:
We specify the users
table.
We update both the name
and email
of the user.
We filter the update to only apply to the active user with id = 1
.
We return the updated email
after the update is committed.
The CQL::Update
class provides a simple yet powerful interface for building and executing UPDATE
queries in a Crystal application. With chainable methods for setting values, applying conditions, and controlling the output, you can easily handle any update operation.
Whether you are updating single records or large batches, the flexibility of CQL::Update
ensures that your queries remain clean, maintainable, and efficient.
HasMany
Relationship GuideIn this guide, we'll focus on the HasMany
relationship using CQL's Active Record syntax. This describes a one-to-many connection between models.
HasMany
Relationship?The HasMany
relationship indicates that one entity (a record) is related to multiple other entities. For example, a Post can have many Comments. This relationship is a one-to-many mapping between two entities.
In a blogging system:
A Post can have many Comments.
Each Comment belongs to one Post.
This is a common one-to-many relationship where one post can have multiple comments, but each comment refers to only one post.
We'll define the posts
and comments
tables in the schema using CQL's DSL.
posts table: Stores post details like title
, body
, and published_at
.
comments table: Stores comment details with a foreign key post_id
that references the posts
table.
Let's db_context the Post
and Comment
models and establish the HasMany
and BelongsTo
relationships in CQL.
The has_many :comments, Comment, foreign_key: :post_id
association in the Post
model defines that each post can have multiple comments. The comments
table must have a post_id
column.
The belongs_to :post, Post, foreign_key: :post_id
in the Comment
model links each comment back to its post.
HasMany
CollectionWhen you access a has_many
association (e.g., post.comments
), you get a collection proxy object that offers several methods to query and manipulate the associated records.
The collection is enumerable and provides methods to access its records.
If the database might have changed, reload the collection:
The has_many
macro generates reload_{{association_name}}
(e.g., reload_comments
).
Deleting a specific comment from the association (and database):
Note: The delete
method on the collection typically removes the record from the database.
Clearing the association (deletes all associated comments):
clear
usually implies deleting the associated records from the database. Be cautious with this method.
If you delete the parent record (post.delete
), associated comments are not automatically deleted unless cascade: true
was specified in the has_many
definition or database-level cascade rules are in place.
To avoid N+1 query problems when loading many posts and their comments, use eager loading:
includes(:comments)
tells CQL to fetch all comments for the retrieved posts in a separate, optimized query.
In this guide, we've explored the HasMany
relationship in CQL. We covered:
Defining Post
and Comment
models with has_many
and belongs_to
associations, including the foreign_key
option.
Interacting with the has_many
collection using methods like create
, build
, <<
, all
, find_by
, exists?
, size
, delete
, and clear
.
Eager loading associations with includes
.
In the next guide, we'll build upon this ERD and cover the ManyToMany
relationship.
AcmeDB = CQL::Schema.define(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :posts do
primary :id, Int64, auto_increment: true
text :title
text :body
timestamp :published_at
end
table :comments do
primary :id, Int64, auto_increment: true
bigint :post_id
text :body
end
end
struct Post
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :posts
property id : Int64?
property title : String
property body : String
property published_at : Time?
# Initializing a new post
def initialize(@title : String, @body : String, @published_at : Time? = Time.utc)
end
# Association: A Post has many Comments
# The `foreign_key` option specifies the column on the `comments` table
# that references the `posts` table.
has_many :comments, Comment, foreign_key: :post_id
end
struct Comment
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :comments
property id : Int64?
property post_id : Int64?
property body : String
# Initializing a comment. Post can be associated later or by the collection.
def initialize(@body : String, @post_id : Int64? = nil)
end
# Association: A Comment belongs to one Post
belongs_to :post, Post, foreign_key: :post_id
end
# Fetch or create a Post
post = Post.find_or_create_by(title: "HasMany Guide Post", body: "Content for HasMany.")
post.save! if !post.persisted? # Ensure post is saved and has an ID
# Option 1: Using `create` on the collection (builds, sets FK, saves, adds to collection)
comment1 = post.comments.create(body: "First comment via create!")
puts "Comment 1: #{comment1.body}, Post ID: #{comment1.post_id}"
# Option 2: Using `build` on the collection (builds, sets FK, but does NOT save yet)
comment2 = post.comments.build(body: "Second comment via build.")
# comment2 is now associated with post (comment2.post_id == post.id) but not saved.
puts "Comment 2 (unsaved): #{comment2.body}, Post ID: #{comment2.post_id}"
comment2.save!
puts "Comment 2 (saved): #{comment2.id}"
# Option 3: Using `<<` operator (creates a new record from an instance and saves it)
# The Comment instance should ideally not have post_id set if `<<` handles it,
# or it should match the parent post. Behavior may vary; `create` is often clearer.
comment3 = Comment.new(body: "Third comment via << operator.")
post.comments << comment3 # This will save comment3 and associate it.
puts "Comment 3: #{comment3.body}, ID: #{comment3.id}, Post ID: #{comment3.post_id}"
# Option 4: Manual creation and association (less common for adding to existing parent)
# comment4 = Comment.new(body: "Fourth comment, manual.", post_id: post.id.not_nil!)
# comment4.save!
# post.comments.reload # Important to see it in the cached collection
# Fetch the post again to ensure a clean comments collection or use post.comments.reload
loaded_post = Post.find!(post.id.not_nil!)
puts "\nComments for Post: '#{loaded_post.title}'"
# Iterating through the collection (implicitly loads if not already loaded)
loaded_post.comments.each do |comment|
puts "- #{comment.body} (ID: #{comment.id})"
end
# Get all comments as an array
all_comments_array = loaded_post.comments.all
puts "Total comments in array: #{all_comments_array.size}"
# Find a specific comment within the association
found_comment = loaded_post.comments.find_by(body: "First comment via create!")
if c = found_comment
puts "Found comment by body: #{c.id}"
end
# Check for existence
is_present = loaded_post.comments.exists?(body: "Second comment via build.")
puts "Does 'Second comment via build.' exist? #{is_present}"
# Get size and check if empty
puts "Number of comments: #{loaded_post.comments.size}"
puts "Are there any comments? #{!loaded_post.comments.empty?}"
# Get first comment
first_comment = loaded_post.comments.first
puts "First comment body: #{first_comment.try(&.body)}"
# Get array of IDs
comment_ids = loaded_post.comments.ids
puts "Comment IDs: #{comment_ids}"
loaded_post.comments.reload # Fetches fresh data from DB
# or for a specific association on a model instance:
# loaded_post.reload_comments # if such a specific reloader is generated by has_many macro
comment_to_delete = loaded_post.comments.find_by(body: "Third comment via << operator.")
if ctd = comment_to_delete
if loaded_post.comments.delete(ctd) # Pass instance or its ID
puts "Successfully deleted comment ID: #{ctd.id}"
else
puts "Could not delete comment ID: #{ctd.id}"
end
end
# Verify deletion
puts "Comment count after delete: #{loaded_post.comments.size}"
# First, add some comments if cleared previously
loaded_post.comments.create(body: "Temp comment 1 for clear test")
loaded_post.comments.create(body: "Temp comment 2 for clear test")
puts "Comments before clear: #{loaded_post.comments.size}"
loaded_post.comments.clear # Deletes all comments associated with this post from the database
puts "Comments after clear: #{loaded_post.comments.size}" # Should be 0
# Fetches all posts and their associated comments in a more optimized way (typically 2 queries)
posts_with_comments = Post.query.includes(:comments).all(Post)
posts_with_comments.each do |p|
puts "Post: #{p.title} has #{p.comments.size} comments (already loaded):"
p.comments.each do |c| # Accesses the already loaded comments
puts " - #{c.body}"
end
end
The Queryable
module in CQL provides a powerful, type-safe, and chainable interface for constructing SQL queries using idiomatic Crystal code. It enables you to filter, sort, join, and aggregate records with a fluent API, abstracting away raw SQL while retaining full expressiveness and performance.
This guide covers how to use the query interface provided by CQL Active Record models and repositories, including all major query methods and best practices.
You can build queries directly on your model or repository. Most query methods return a chainable query object, allowing you to compose complex queries step by step.
# Find all users named "Alice"
alice_users = User.where(name: "Alice").all
# Find the first user with a specific email
user = User.where(email: "alice@example.com").first
user_repo = UserRepository.new
# Find all users with a given domain
gmail_users = user_repo.query.where { email.like("%@gmail.com") }.all(User)
CQL provides a rich set of chainable query methods that allow you to build complex queries in a type-safe and expressive way. Each method returns a query object that can be further chained with additional methods.
where
The where
method is used to filter records based on conditions. It supports both simple key-value pairs and block-based conditions.
# Simple key-value conditions
User.where(name: "Alice", active: true)
# Block-based conditions with operators
User.where { age > 18 }
User.where { (name.like("A%")) & (age >= 21) }
# Multiple conditions
User.where(name: "Alice").where(active: true)
order
Sort records using the order
method. You can specify multiple columns and sort directions.
# Single column ordering
User.order(:name) # ASC by default
User.order(name: :desc) # Explicit DESC
# Multiple columns
User.order(:created_at, name: :desc)
# Using block syntax
User.order { created_at.desc }
limit
and offset
Control the number of records returned and paginate through results.
# Get first 10 records
User.limit(10)
# Skip first 20 records and get next 10
User.offset(20).limit(10)
# Common pagination pattern
page = 2
per_page = 10
User.offset((page - 1) * per_page).limit(per_page)
select
Specify which columns to retrieve from the database.
# Select specific columns
User.select(:id, :name, :email)
# Using block syntax
User.select { [id, name, email] }
# With aliases
User.select { [id.as("user_id"), name.as("user_name")] }
group_by
Group records by one or more columns, often used with aggregate functions.
# Group by single column
User.group_by(:role)
# Multiple columns
User.group_by(:role, :status)
# With block syntax
User.group_by { [role, status] }
join
Perform SQL joins between tables. CQL supports various join types and provides a type-safe way to specify join conditions.
# Simple inner join
User.join(Post)
# Join with conditions
User.join(Post) { users.id == posts.user_id }
# Multiple joins
User.join(Post)
.join(Comment) { posts.id == comments.post_id }
# Different join types
User.left_join(Post)
User.right_join(Post)
User.full_join(Post)
having
Add conditions to grouped records, typically used with group_by
.
User.group_by(:role)
.having { count(id) > 5 }
distinct
Remove duplicate records from the result set.
User.distinct
User.distinct(:name, :email)
Terminal operations execute the query and return the results. These methods are typically used at the end of a query chain to fetch the data from the database.
all
Returns all records matching the query as an array.
# Get all users
users = User.all
# Get all active users ordered by name
active_users = User.where(active: true)
.order(:name)
.all
# Get all users with their posts
users_with_posts = User.join(Post)
.select { [users.*, posts.*] }
.all
first
and first!
Returns the first record matching the query. first
returns nil
if no record is found, while first!
raises an error.
# Get the first user (returns nil if none found)
first_user = User.first
# Get the first user (raises error if none found)
first_user = User.first!
# Get the first active user ordered by creation date
oldest_active = User.where(active: true)
.order(:created_at)
.first
find
and find!
Find a record by its primary key. find
returns nil
if not found, while find!
raises an error.
# Find user by ID (returns nil if not found)
user = User.find(1)
# Find user by ID (raises error if not found)
user = User.find!(1)
# Find with additional conditions
active_user = User.where(active: true).find(1)
count
Returns the number of records matching the query.
# Count all users
total_users = User.count
# Count active users
active_count = User.where(active: true).count
# Count users by role
role_counts = User.group_by(:role)
.select { [role, count(id).as("count")] }
.all
exists?
Checks if any records match the query.
# Check if any users exist
has_users = User.exists?
# Check if any active users exist
has_active = User.where(active: true).exists?
# Check if a specific user exists
user_exists = User.where(email: "alice@example.com").exists?
pluck
Returns an array of values for the specified columns.
# Get all user names
names = User.pluck(:name)
# Get multiple columns
name_emails = User.pluck(:name, :email)
# With conditions
active_names = User.where(active: true)
.pluck(:name)
sum
, average
, minimum
, maximum
Aggregate functions that can be used to calculate statistics.
# Calculate total age of all users
total_age = User.sum(:age)
# Calculate average age
avg_age = User.average(:age)
# Find minimum and maximum ages
min_age = User.minimum(:age)
max_age = User.maximum(:age)
# With conditions
active_avg_age = User.where(active: true)
.average(:age)
find_each
and find_in_batches
Process records in batches to handle large datasets efficiently.
# Process users one at a time
User.find_each(batch_size: 1000) do |user|
# Process each user
process_user(user)
end
# Process users in batches
User.find_in_batches(batch_size: 1000) do |users|
# Process batch of users
process_users(users)
end
CQL's query interface allows you to build complex queries by combining multiple query methods. This section demonstrates common patterns and best practices for composing sophisticated queries.
Combine multiple conditions using logical operators and complex expressions.
# Complex where conditions
User.where { (age >= 18) & (active == true) }
.where { (name.like("A%")) | (email.like("%@gmail.com")) }
# Nested conditions
User.where { (role == "admin") & ((status == "active") | (status == "pending")) }
# Using IN clauses
User.where { role.in(["admin", "moderator"]) }
.where { status.in(["active", "pending"]) }
Combine joins with filtering and ordering.
# Join with filtering on both tables
User.join(Post) { users.id == posts.user_id }
.where { (users.active == true) & (posts.published == true) }
.order { [users.name, posts.created_at.desc] }
# Multiple joins with conditions
User.join(Post) { users.id == posts.user_id }
.join(Comment) { posts.id == comments.post_id }
.where { comments.created_at > 1.week.ago }
.select { [users.*, posts.title, count(comments.id).as("comment_count")] }
.group_by { [users.id, posts.id] }
Combine grouping with aggregations and having clauses.
# User statistics by role
User.group_by(:role)
.select { [role, count(id).as("total"),
average(age).as("avg_age"),
maximum(age).as("max_age")] }
.having { count(id) > 5 }
.order { count(id).desc }
# Post statistics with user information
User.join(Post) { users.id == posts.user_id }
.group_by { [users.id, users.name] }
.select { [users.name,
count(posts.id).as("post_count"),
average(posts.view_count).as("avg_views")] }
.having { count(posts.id) >= 3 }
Combine pagination with other query methods.
# Paginated search with sorting
def search_users(query, page = 1, per_page = 20)
User.where { (name.like("%#{query}%")) |
(email.like("%#{query}%")) }
.order(:name)
.offset((page - 1) * per_page)
.limit(per_page)
end
# Paginated join with conditions
def recent_user_posts(user_id, page = 1, per_page = 10)
User.join(Post) { users.id == posts.user_id }
.where(users.id: user_id)
.where { posts.created_at > 1.month.ago }
.order { posts.created_at.desc }
.offset((page - 1) * per_page)
.limit(per_page)
end
Use subqueries for complex filtering and data retrieval.
# Users with more than 5 posts
User.where { id.in(
Post.select(:user_id)
.group_by(:user_id)
.having { count(id) > 5 }
) }
# Users with recent activity
User.where { id.in(
Post.select(:user_id)
.where { created_at > 1.week.ago }
.union(
Comment.select(:user_id)
.where { created_at > 1.week.ago }
)
) }
Use CTEs for complex queries that need to reference the same subquery multiple times.
# Users with their post and comment counts
User.with(:user_stats) {
User.select { [
id,
count(posts.id).as("post_count"),
count(comments.id).as("comment_count")
] }
.left_join(Post) { users.id == posts.user_id }
.left_join(Comment) { users.id == comments.user_id }
.group_by(:id)
}
.select { [users.*, user_stats.*] }
.join(:user_stats) { users.id == user_stats.id }
Define reusable query scopes for common query patterns.
struct User
include CQL::ActiveRecord::Model(Int32)
# ... other code ...
def self.active
where(active: true)
end
def self.admins
where(role: "admin")
end
def self.recently_created
where { created_at > 1.week.ago }
end
def self.with_posts
join(Post) { users.id == posts.user_id }
end
end
# Using scopes
User.active
.admins
.recently_created
.with_posts
.order(:name)
Following these best practices will help you write more efficient, maintainable, and performant queries in CQL.
Select Only Needed Columns
# Instead of
User.all
# Use
User.select(:id, :name, :email).all
Use Appropriate Indexes
Add indexes for frequently queried columns
Index foreign keys and columns used in WHERE clauses
Consider composite indexes for commonly combined conditions
Avoid N+1 Queries
# Instead of
users = User.all
users.each do |user|
posts = user.posts.all # N+1 queries!
end
# Use
User.join(Post)
.select { [users.*, posts.*] }
.all
Use Batch Processing for Large Datasets
# Instead of
User.all.each do |user|
process_user(user)
end
# Use
User.find_each(batch_size: 1000) do |user|
process_user(user)
end
Use Query Scopes for Reusable Logic
struct User
include CQL::ActiveRecord::Model(Int32)
def self.active
where(active: true)
end
def self.recent
where { created_at > 1.week.ago }
end
end
# Usage
User.active.recent
Extract Complex Queries to Methods
struct User
def self.search_by_name_or_email(query)
where { (name.like("%#{query}%")) |
(email.like("%#{query}%")) }
end
def self.with_post_counts
left_join(Post) { users.id == posts.user_id }
.group_by(:id)
.select { [users.*, count(posts.id).as("post_count")] }
end
end
Use Meaningful Names for Complex Queries
def find_active_users_with_recent_posts
User.active
.join(Post) { users.id == posts.user_id }
.where { posts.created_at > 1.week.ago }
.distinct
end
Use Parameterized Queries
# Instead of
User.where { name == "#{user_input}" }
# Use
User.where(name: user_input)
Validate Input Before Querying
def search_users(query)
return User.none if query.blank?
User.where { name.like("%#{query}%") }
end
Handle Edge Cases
def find_user_by_email(email)
return nil if email.blank?
User.where(email: email.downcase.strip).first
end
Test Query Scopes
describe User do
it "finds active users" do
active_user = User.create(active: true)
inactive_user = User.create(active: false)
expect(User.active).to contain(active_user)
expect(User.active).not_to contain(inactive_user)
end
end
Test Complex Queries
describe User do
it "finds users with recent posts" do
user = User.create
old_post = Post.create(user: user, created_at: 2.weeks.ago)
recent_post = Post.create(user: user, created_at: 3.days.ago)
result = User.with_recent_posts
expect(result).to contain(user)
end
end
Use to_sql
for Query Inspection
query = User.where(active: true).order(:name)
puts query.to_sql # Prints the generated SQL
Log Slow Queries
def self.log_slow_queries
start_time = Time.monotonic
result = yield
duration = Time.monotonic - start_time
if duration > 1.second
Log.warn { "Slow query detected: #{duration.total_seconds}s" }
end
result
end
CQL's Active Record provides basic pagination functionality directly on your models, allowing you to easily retrieve records in chunks or pages. This is essential for handling large datasets efficiently, especially in web applications.
The pagination methods are available as class methods on any model that includes CQL::ActiveRecord::Model(Pk)
.
page(page_number, per_page \\= 10)
The page
method is the primary way to retrieve a specific page of records.
page_number : Int32
: The desired page number (1-indexed).
per_page : Int32
: The number of records to include on each page. Defaults to 10
.
It calculates the necessary offset
and applies a limit
to the query.
Example:
How it works internally:
The method essentially performs:query.limit(per_page).offset((page_number - 1) * per_page).all(ModelName)
per_page(num_records)
The per_page
method, when used as a standalone class method, sets a limit on the number of records returned. It effectively retrieves the first page of records with the specified num_records
count.
num_records : Int32
: The number of records to retrieve.
Example:
How it works internally:
The method performs:query.limit(num_records).all(ModelName)
Note on per_page
:
While per_page
can be called directly on the model class, its name might suggest it's primarily a modifier for other pagination logic (which isn't directly supported by chaining these specific class methods). In most common pagination scenarios, the page(page_number, per_page)
method is more comprehensive as it handles both the page number and the items per page.
Using per_page(n)
is equivalent to page(1, per_page: n)
.
Pagination methods are applied to the model's default query scope. If you need to paginate a filtered set of records, you would typically chain pagination methods onto a CQL::Query
object obtained via YourModel.query
:
The standalone User.page
and User.per_page
methods are convenient for simple, direct pagination on an entire table. For more complex scenarios, building the query and then applying limit
and offset
manually (as shown above) provides greater flexibility, and is what User.page
does internally.
It's important to note that the Pagination
module in active_record/pagination.cr
provides these as class methods directly on the model. If you need to paginate a more complex query chain, you'd apply .limit()
and .offset()
to the query object itself.
# Assuming you have a User model defined:
struct User
includes CQL::ActiveRecord::Model(Int64)
db_context YourDB, :users
# ... properties ...
end
# Get the first page, 10 users per page (default per_page)
page1_users = User.page(1)
# Get the second page, 10 users per page
page2_users = User.page(2)
# Get the third page, with 5 users per page
page3_users_custom_per_page = User.page(3, per_page: 5)
puts "Page 1 Users (#{page1_users.size} users):"
page1_users.each do |user|
puts "- ID: #{user.id}, Name: #{user.name}"
end
puts "\\nPage 3 Users, 5 per page (#{page3_users_custom_per_page.size} users):"
page3_users_custom_per_page.each do |user|
puts "- ID: #{user.id}, Name: #{user.name}"
end
# Get the first 5 users
first_5_users = User.per_page(5)
puts "\\nFirst 5 Users (#{first_5_users.size} users):"
first_5_users.each do |user|
puts "- ID: #{user.id}, Name: #{user.name}"
end
# Get page 2 of active users, 5 per page
active_users_page2 = User.query
.where(active: true)
.order(created_at: :desc)
.limit(5) # This is per_page
.offset((2 - 1) * 5) # This is (page_number - 1) * per_page
.all(User)
puts "\\nActive Users, Page 2, 5 per page (#{active_users_page2.size} users):"
active_users_page2.each do |user|
puts "- ID: #{user.id}, Name: #{user.name}, Active: #{user.active}"
end
Beyond basic CRUD operations, CQL Active Record models offer methods to understand and manage their persistence state.
This guide assumes you have a model defined, for example:
struct User
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :users
property id : Int64?
# ... other properties ...
end
It's often useful to know if a model instance represents a record that already exists in the database or if it's a new record that hasn't been saved yet.
persisted?
The persisted?
instance method returns true
if the record is considered to be saved in the database, and false
otherwise. Typically, this means the instance has a non-nil primary key (id
).
# New, unsaved record
new_user = User.new(name: "Temp User", email: "temp@example.com")
puts "Is new_user persisted? #{new_user.persisted?}" # => false
puts "new_user ID: #{new_user.id.inspect}" # => nil (assuming id is Int64? and auto-generated)
# Save the record
if new_user.save
puts "User saved."
puts "Is new_user persisted now? #{new_user.persisted?}" # => true
puts "new_user ID after save: #{new_user.id.inspect}" # => 1 (or some other generated ID)
else
puts "Failed to save new_user."
end
# Loaded record
existing_user = User.find?(new_user.id.not_nil!) # Find the user we just saved
if existing_user
puts "Is existing_user persisted? #{existing_user.persisted?}" # => true
puts "existing_user ID: #{existing_user.id.inspect}"
end
Use persisted?
to:
Differentiate between new and existing records in forms or views.
Decide whether an update or an insert operation is appropriate in more manual scenarios.
Conditionally execute logic based on whether a record is already in the database.
new_record?
(Conceptual / Alias for !persisted?
)Some ORMs provide a new_record?
method, which is typically the opposite of persisted?
. While CQL's core Persistence
module might not explicitly define new_record?
, you can achieve the same by checking !instance.persisted?
.
user = User.new(name: "Another Temp", email: "another@example.com")
if !user.persisted?
puts "This is a new record (not persisted)."
end
Sometimes, the data for a record in your application might become stale if the corresponding row in the database has been changed by another process or a different part of your application. The reload!
method allows you to refresh an instance's attributes from the database.
reload!
The reload!
instance method fetches the latest data from the database for the current record (identified by its primary key) and updates the instance's attributes in place.
If the record no longer exists in the database (e.g., it was deleted by another process), reload!
will typically raise a DB::NoResultsError
or a similar RecordNotFound
exception.
# Assume user_jane exists and her email is "jane.doe@example.com"
user_jane = User.find_by!(email: "jane.doe@example.com")
puts "Initial name for Jane: #{user_jane.name}"
# Simulate an external update to the database for this user
# (In a real scenario, this would happen outside this code flow)
# For example, directly via SQL: UPDATE users SET name = 'Jane Updated Externally' WHERE id = user_jane.id;
# Now, reload user_jane to get the latest data
try
user_jane.reload!
puts "Reloaded name for Jane: #{user_jane.name}" # Should show "Jane Updated Externally"
# If user_jane was deleted externally before reload!
# user_jane.reload! # This would raise DB::NoResultsError
rescue DB::NoResultsError
puts "Record for user '#{user_jane.name}' (ID: #{user_jane.id}) no longer exists in the database."
rescue ex : Exception
puts "An error occurred during reload: #{ex.message}"
end
Use reload!
when:
You need to ensure you are working with the most up-to-date version of a record, especially before performing critical operations or displaying sensitive data.
You suspect an instance's state might be out of sync with the database due to concurrent operations.
Understanding these persistence details helps in managing the lifecycle and state of your Active Record model instances effectively.
Here are some common issues you might encounter while using CQL and how to resolve them.
NoMethodError
when queryingEnsure that your table and columns are correctly defined in the schema. For example:
table :users do
primary :id, Int64
column :name, String
end
If you're querying a column that doesn't exist, CQL will raise a NoMethodError
.
Ensure that any errors raised inside the transaction block are properly handled. If an error occurs, the transaction will be rolled back automatically.
In this guide, we'll cover the BelongsTo
relationship using CQL's Active Record syntax. We'll start with an Entity-Relationship Diagram (ERD) to illustrate how this relationship works and continuously build upon this diagram as we introduce new relationships in subsequent guides.
BelongsTo
Relationship?The BelongsTo
association in a database indicates that one entity (a record) refers to another entity by holding a foreign key to that record. For example, a Comment
belongs to a Post
, and each comment references the Post
it is associated with by storing the post_id
as a foreign key.
Let's say you have a blog system where:
A Post can have many Comments.
A Comment belongs to one Post.
We'll start by implementing the BelongsTo
relationship from the Comment
to the Post
.
We'll first define the posts
and comments
tables using CQL's schema DSL.
codeAcmeDB = CQL::Schema.define(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :posts do
primary
text :title
text :body
timestamp :published_at
end
table :comments do
primary
bigint :post_id
text :body
end
end
posts table: Contains the blog post data (title, body, and published date).
comments table: Contains the comment data and a foreign key post_id
which references the posts
table.
Next, we'll define the Post
and Comment
structs in CQL.
struct Post
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :posts
property id : Int64?
property title : String
property body : String
property published_at : Time?
# Initializing a new post with title, body, and optional published_at
def initialize(@title : String, @body : String, @published_at : Time? = Time.utc)
end
end
struct Comment
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :comments
property id : Int64?
property body : String
property post_id : Int64?
# Initializing a comment with a body. Post can be associated later.
def initialize(@body : String, @post_id : Int64? = nil)
end
# Association: Each Comment belongs to one Post
belongs_to :post, Post, foreign_key: :post_id
end
In the Comment
model, we specify the belongs_to :post, Post, foreign_key: :post_id
association. This links each comment to its parent post. The Comment
model must have a post_id
attribute (matching the foreign_key
option) that stores the id
of the associated Post
.
Now that we have defined the Post
and Comment
models with a belongs_to
relationship, let's see how to create and query records in CQL.
Option 1: Create Parent, then Child
# 1. Create and save the parent Post first
post = Post.new(title: "My First Blog Post", body: "This is the body of the post.")
post.save!
# 2. Create the Comment and associate it
# a) By setting the foreign key directly:
comment1 = Comment.new(body: "Great post via direct FK!")
comment1.post_id = post.id # Assign the foreign key
comment1.save!
puts "Comment 1 associated with Post ID: #{comment1.post_id}, Post title: #{comment1.post.try(&.title)}"
# b) By using the association setter:
comment2 = Comment.new(body: "Awesome article via association setter!")
comment2.post = post # Assign the Post instance to the association
comment2.save!
puts "Comment 2 associated with Post ID: #{comment2.post_id}, Post title: #{comment2.post.try(&.title)}"
Option 2: Create Child and Associated Parent Simultaneously (if needed)
If you have a Comment
instance and want to create its Post
at the same time (less common for belongs_to
primary creation flow but possible via association methods):
new_comment = Comment.new(body: "A comment for a brand new post.")
# This creates a new Post, saves it, and associates it with new_comment
created_post = new_comment.create_post(title: "Post Created Via Comment", body: "Content for post created via comment.")
new_comment.save! # Save the comment itself which now has the post_id populated
puts "New comment ID: #{new_comment.id}, associated Post ID: #{new_comment.post_id}"
puts "Title of post created via comment: #{created_post.title}"
puts "Comment's post title: #{new_comment.post.try(&.title)}"
Note: create_association
(like create_post
) will create and save the associated object (Post
) and set the foreign key on the current object (new_comment
). The current object itself (new_comment
) still needs to be saved if it's new.
Option 3: Build Associated Parent (without saving parent yet)
yet_another_comment = Comment.new(body: "Comment with a built post.")
# This builds a new Post instance but does not save it to the DB yet.
# The foreign key on `yet_another_comment` is not set by `build_post`.
built_post = yet_another_comment.build_post(title: "Built, Not Saved Post", body: "Body of built post.")
# You would then save `built_post` and then `yet_another_comment` after setting association.
# built_post.save!
# yet_another_comment.post = built_post
# yet_another_comment.save!
puts "Built post title: #{built_post.title} (not yet saved)"
Once we have a comment, we can retrieve the associated post using the belongs_to
association.
crystalCopy code# Fetch the comment
comment = Comment.find(1)
# Fetch the associated post
post = comment.post
puts post.title # Outputs: "My First Blog Post"
In this example, comment.post
will fetch the Post
associated with that Comment
.
In this guide, we've covered the basics of the belongs_to
relationship in CQL. We:
Defined the Post
and Comment
tables in the schema.
Created the corresponding models, specifying the belongs_to
relationship in the Comment
model.
Showed how to create and query records using the belongs_to
association.
In the next guides, we'll build on this ERD and introduce other types of relationships like has_one
, has_many
, and many_to_many
. Stay tuned for the next part where we'll cover the has_many
relationship!
Feel free to play around with this setup and extend the models or experiment with more queries to familiarize yourself with CQL's Active Record capabilities.
This page answers common questions about Crystal Query Language (CQL) and its Active Record implementation.
Q: What is CQL (Crystal Query Language)?
A: CQL is an Object-Relational Mapping (ORM) library for the Crystal programming language. It provides an abstraction over SQL databases, enabling developers to define and interact with relational data using Crystal objects and a type-safe query builder, rather than writing raw SQL queries directly for most operations.
Q: What are the key features of CQL?
A: Key features typically include:
Type-safe query building leveraging Crystal's static type system.
A macro-powered DSL for defining models and their mappings.
Support for multiple database adapters (e.g., PostgreSQL, MySQL, SQLite).
Migrations system for managing database schema changes.
An Active Record pattern implementation for model interaction (and flexibility for other patterns).
Q: How do I install CQL?
A: You can install CQL by adding it to your project's shard.yml
file:
dependencies:
cql:
github: azutoolkit/cql # Or the appropriate source for your CQL version
version: "~> x.y.z" # Specify the version you are using
Then, run shards install
in your terminal to download and install the dependency.
For more setup details, see the Active Record Setup Guide.
Q: Which databases does CQL support?
A: CQL is designed to support major relational databases through Crystal DB drivers. Commonly supported databases include PostgreSQL, MySQL, and SQLite. Check the specific CQL version documentation for the most up-to-date list and any driver requirements.
Q: What is the Active Record pattern in CQL?
A: The Active Record pattern maps database tables to Crystal structs (models). Each instance of a model corresponds to a row in the table. This pattern provides methods directly on the model and its instances for database interaction (CRUD operations, queries, etc.).
For a conceptual overview, see Active Record Pattern Concept.
For a detailed guide on using it with CQL, see Active Record with CQL Guide.
Q: How do I define an Active Record model in CQL?
A: You define a model by creating a Crystal struct
, including CQL::ActiveRecord::Model(PrimaryKeyType)
, and using the db_context
macro to link it to a database table and context.
Learn more in the Defining Models Guide.
Q: How do I perform basic Create, Read, Update, and Delete (CRUD) operations?
A: CQL Active Record provides intuitive methods like new
/save
, create!
, find?
, find_by!
, update!
, and delete!
directly on your models and their instances.
For comprehensive examples, see the CRUD Operations Guide.
Q: How can I build more complex queries?
A: CQL offers a chainable query interface. You can start with Model.query
or methods like Model.where(...)
and then chain further conditions like .order()
, .limit()
, .join()
, etc.
Explore the Querying Guide for detailed information.
Q: How do I handle database relationships (e.g., has_many
, belongs_to
)?
A: CQL Active Record uses macros like has_many
, belongs_to
, has_one
, and many_to_many
to define associations between models.
See the Relations section in the main Active Record guide for links to detailed guides on each relationship type:
Q: How are validations handled in CQL Active Record?
A: You can define validations in your model to ensure data integrity (e.g., presence, length, format). Invalid records will not be saved, and errors can be inspected on the model instance.
Refer to the Validations Guide.
Q: What are callbacks and how do I use them?
A: Callbacks are methods that get triggered at specific points in a model's lifecycle (e.g., before_save
, after_create
). They allow you to run custom logic automatically.
Learn how to use them in the Callbacks Guide.
Q: What's the difference between save
and save!
(or create
and create!
)?
A: Methods ending with !
(bang methods) typically raise an exception if the operation fails (e.g., a validation error occurs or the record isn't found). Methods without !
usually return false
or nil
on failure, allowing you to handle errors programmatically without a begin/rescue
block for common cases.
Example: user.save
returns true
or false
. user.save!
returns true
or raises an exception (e.g., CQL::Errors::RecordInvalid
).
Q: How does CQL handle database migrations for Active Record models?
A: CQL includes a migration system where you define schema changes in Crystal classes (e.g., creating tables, adding columns). These migrations can be run to update your database schema in a version-controlled manner.
For details, see the Database Migrations Guide for Active Record.
For a general overview of migrations in CQL, see Core Concepts: Migrations.
Q: Does CQL Active Record support database transactions?
A: Yes, database transactions are crucial for ensuring data consistency, especially when multiple database operations need to succeed or fail together. CQL typically provides a way to manage transactions.
See the Transaction Management Guide for more details (Note: Ensure this guide specifically covers Active Record transaction patterns if applicable, or link to a more general CQL transaction guide if that's more appropriate).
Q: Where can I find more detailed documentation on Active Record features?
A: The primary resource is the Active Record with CQL main guide, which links to all specialized guides covering different aspects of the Active Record implementation.
Q: I think I found a bug or want to suggest a feature. How do I proceed?
A: Typically, you would report issues or suggest features via the GitHub repository for CQL (e.g., azutoolkit/cql
or the specific fork/version you are using). Look for an "Issues" tab to see if similar issues exist or to create a new one. Contributions via Pull Requests are also often welcome after discussing the proposed changes.
Q: Is there a community or forum for CQL users?
A: Check the official CQL repository or related Crystal community channels (like the Crystal forum, Discord, or Gitter) for discussions, help, and community support related to CQL.
If your question isn't answered here, please check the other documentation sections or consider reaching out to the CQL community.
An overview of CQL's Active Record capabilities for defining models, interacting with your database, managing data integrity, and more.
This guide provides a comprehensive overview of Crystal Query Language (CQL)'s Active Record implementation. Active Record is a design pattern that connects database tables to classes (or structs in Crystal), allowing you to interact with your data through objects and methods rather than raw SQL queries.
CQL's Active Record module offers a powerful and intuitive way to manage your database records, inspired by established ORMs while leveraging Crystal's type safety and performance.
This central README provides a high-level introduction. For in-depth information on specific aspects of CQL Active Record, please refer to the following guides:
Setup and Prerequisites: Initial configuration for using CQL and Active Record. (Covered below)
Defining Models: Learn how to define your Active Record models, map them to database tables, specify primary keys, and work with attributes.
CRUD Operations: Detailed guide on creating, reading, updating, and deleting records using Active Record methods.
Querying: Explore the powerful query interface, including direct finders, chainable queries, aggregations, and scopes.
Transactions: Ensure data integrity by using database transactions for multi-step operations.
Persistence Details: Understand how to check if a record is persisted and how to reload its data from the database.
Validations: Ensure data integrity by defining and using model validations.
Callbacks: Hook into the lifecycle of your models to trigger logic at specific events (e.g., before save, after create).
Relations: Define and use associations between models:
many_to_many
(covers has_and_belongs_to_many
)
Database Migrations: Manage your database schema changes over time.
Scopes: Define reusable query constraints for cleaner and more readable code.
Pagination: Easily paginate query results.
Before getting started, ensure you have the following:
Crystal language installed (latest stable version recommended).
A supported relational database (e.g., PostgreSQL, MySQL) set up and accessible.
CQL added to your Crystal project.
Include CQL in your project's shard.yml
:
dependencies:
cql:
github: azutoolkit/cql # Or the appropriate source for your CQL version
version: "~> x.y.z" # Specify the version you are using
Then, run shards install
to download and install the dependency.
You need to configure CQL to connect to your database. This is typically done by setting a database URL and opening a connection. You might also define a database context for your application.
require "cql"
# Example: Define your database connection URL (replace with your actual credentials)
# For PostgreSQL:
ENV_DB_URL = ENV["DATABASE_URL"]? || "postgres://username:password@localhost:5432/myapp_development"
# Define a database context. This is often a class or module that your models will reference.
# The name `AcmeDB` is used as a placeholder in these guides.
module AcmeDB
# Establishes and memoizes the database connection.
def self.db
@@db ||= DB.open(ENV_DB_URL)
end
# Optional: A method to close the connection if needed during shutdown or testing.
def self.close_db
@@db.try(&.close)
@@db = nil
end
end
# Ensure your models can reference this context, e.g.:
struct User
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :users
# ...
end
Note: The exact mechanism for defining your database context (AcmeDB
in the example) and making it accessible to your models should align with CQL's specific API and your application structure. Refer to CQL's core documentation for advanced database connection management, pooling, and context configuration.
Models are Crystal structs
including CQL::ActiveRecord::Model(PkType)
and use db_context
to link to a table.
See the full Defining Models Guide for details on attributes, primary keys, and more.
CQL provides intuitive methods for creating, reading, updating, and deleting records (e.g., save
, create!
, find?
, find_by!
, update!
, delete!
).
Explore the CRUD Operations Guide for comprehensive examples.
Fetch records using direct finders or build complex queries with a chainable interface (.where
, .order
, .limit
, etc.).
Dive into the Querying Guide for all query-building capabilities.
Maintain data integrity with ACID-compliant database transactions. CQL provides both model-level transaction support and a service objects pattern for complex operations.
# Basic transaction usage
BankAccount.transaction do |tx|
account = BankAccount.find(1)
account.balance -= 100
account.save!
Transaction.create!(
amount: 100,
transaction_type: "withdrawal",
from_account_id: account.id,
created_at: Time.utc
)
# All operations succeed or fail together
end
Learn more in the Transactions Guide for maintaining data integrity across multiple operations.
Ensure data integrity with built-in or custom validation rules triggered before saving records.
Learn more in the Validations Guide.
Execute custom logic at different points in a model's lifecycle (e.g., before_save
, after_create
).
Consult the Callbacks Guide for usage details.
Define associations like belongs_to
, has_many
, has_one
, and many_to_many
to manage relationships between models.
Manage database schema changes systematically using Crystal-based migration files.
See the Database Migrations Guide for how to write and run migrations.
Create reusable query shortcuts to keep your code clean and expressive.
Read the Scopes Guide for defining and using scopes.
Optimistic locking is a concurrency control strategy that allows multiple transactions to proceed without blocking each other, while still preventing conflicts from concurrent updates. CQL provides built-in support for optimistic locking through a version column mechanism.
A version column (e.g., an integer) is added to your database table.
When a record is read, its current version number is also fetched.
When an attempt is made to update the record, the ORM includes a condition in the UPDATE
statement to check if the version number in the database is still the same as when it was fetched.
Simultaneously, the UPDATE
statement also increments this version number.
If the version number in the database has changed (meaning another process updated the record in the meantime), the WHERE
condition (e.g., WHERE id = ? AND version = ?
) will not match any rows. The update will affect zero rows.
The ORM detects that no rows were affected and raises an OptimisticLockError
, signaling that a concurrent update occurred.
The application must then handle this error, typically by reloading the record (to get the new version and latest data) and retrying the update.
Optimistic locking is particularly well-suited for scenarios where:
Conflicts are Rare: You expect data conflicts to be infrequent. If conflicts are very common, the overhead of retrying transactions might outweigh the benefits.
High Concurrency is Required: The application needs to support many users or processes accessing the same data concurrently, and you want to avoid the performance bottlenecks associated with pessimistic locking (which locks records for the duration of a transaction).
Read-Heavy Workloads: Read operations are significantly more frequent than write operations. Optimistic locking doesn't impose any overhead on reads.
User-Facing Applications: In applications where a user might have data форм open for a while before submitting changes, optimistic locking can prevent their changes from unknowingly overwriting updates made by others in the interim.
Disconnected or Stateless Environments: Holding database locks across requests in a stateless web application or a distributed system can be complex or impractical. Optimistic locking provides a way to manage concurrency without long-lived locks.
Cost of Retry is Low: The business logic can tolerate and gracefully handle retries. For example, if updating a product description fails due to a conflict, reloading and trying again is often acceptable.
It's generally not recommended if:
Conflicts are very frequent, as this will lead to many retries and potentially poor performance.
The cost of a failed transaction retry is very high (e.g., involving complex external API calls that are not idempotent).
When defining your table schema, add a version column using the lock_version
method:
table :users do
primary :id, Int64
varchar :name, String
varchar :email, String
lock_version :version # Adds an integer column named 'version' with default value 1
end
The lock_version
method accepts the following parameters:
name
(Symbol): The name of the column (default: :version
). This column should typically be an Int32
or Int64
.
as_name
(String?): An optional alias for the column in queries (rarely needed for version columns).
null
(Bool): Whether the column can be null (default: false
). It's crucial that version columns are not nullable.
default
(DB::Any): The default value for new records (default: 1
).
index
(Bool): Whether to create an index on this column (default: false
). Indexing might be beneficial if you query by version, but it's not strictly necessary for the locking mechanism itself.
When using raw queries, you can use the with_optimistic_lock
method on the Update
object:
# Assuming 'schema' is your CQL::Schema instance
# And we have a record with id=1 and its current version is 5
begin
update_result = CQL::Update.new(schema)
.table(:users)
.set(name: "John Doe", age: 31) # age is an example, ensure your table has it
.where(id: 1)
.with_optimistic_lock(version: 5) # Checks for version 5, will set to 6
.commit
if update_result.rows_affected == 1
puts "Update successful!"
end
rescue CQL::OptimisticLockError
puts "Conflict detected! Record was updated by another process."
# Implement retry logic: reload data and try update again
end
If another process has already updated the record (version is no longer 5), a CQL::OptimisticLockError
will be raised by the .commit
call (or specifically, by the underlying mechanism in CQL::Update
when with_optimistic_lock
is used and rows_affected
is 0).
For Active Record models, include the CQL::ActiveRecord::OptimisticLocking
module and define which column to use with the optimistic_locking
macro:
class User < CQL::ActiveRecord::Base # Assuming Base is your base AR class
include CQL::ActiveRecord::OptimisticLocking
# db_context points to your schema and table
db_context MY_SCHEMA, :users # Replace MY_SCHEMA with your actual schema variable
# Define properties matching your table columns
property id : Int64?
property name : String?
property email : String?
property version : Int32? # This will hold the lock version
# Configure optimistic locking
optimistic_locking version_column: :version
end
This will automatically handle version checking and incrementing when updating records:
user = User.find!(1)
user.name = "Johnathan Doe"
begin
user.update! # Uses optimistic locking automatically
puts "User updated successfully. New version: #{user.version}"
rescue CQL::OptimisticLockError
puts "Failed to update user: Optimistic lock conflict."
# Handle the conflict, e.g., reload and retry
user.reload!
puts "Retrying update after reloading. Current version: #{user.version}"
user.name = "Johnathan Doe" # Re-apply changes if necessary
user.update! # Retry the update
puts "User updated successfully on retry. New version: #{user.version}"
end
If a concurrent update has occurred, user.update!
will raise a CQL::OptimisticLockError
.
When a CQL::OptimisticLockError
is caught, your application needs to decide how to proceed. Common strategies include:
Reload and Retry:
Reload the record from the database to get the latest version and data.
Re-apply the intended changes to the newly reloaded record.
Attempt the update again.
You might want to limit the number of retries to avoid infinite loops in high-contention scenarios.
user = User.find!(1)
max_retries = 3
attempt = 0
loop do
attempt += 1
user.name = "John Doe Updated" # Apply desired changes
begin
user.update!
puts "Update successful!"
break # Exit loop on success
rescue CQL::OptimisticLockError
if attempt >= max_retries
puts "Max retries reached. Could not update record."
# Potentially notify user or log a more persistent error
break
end
puts "Optimistic lock conflict. Reloading and retrying (attempt #{attempt}/#{max_retries})..."
user.reload! # Reloads the record's attributes, including the version
end
end
Inform the User:
If changes are being made through a UI, inform the user that the data has changed since they last viewed it.
Show them the updated data and allow them to decide whether to overwrite, merge their changes, or discard their changes.
Merge Changes (Advanced):
If possible and logical, attempt to merge the conflicting changes. This is highly application-specific and can be complex. For example, if one user changed the product description and another changed the price, these changes might be mergeable.
Use Transactions (Where Appropriate): While optimistic locking handles concurrent updates to a single record, if your operation involves multiple database changes that must be atomic, ensure these are wrapped in a database transaction. Optimistic locking complements, rather than replaces, the need for transactions for atomicity of complex operations.
Handle Conflicts Gracefully: Provide a clear and understandable user experience when conflicts occur. Avoid cryptic error messages. Explain that the data was changed by someone else and suggest next steps.
Selective Usage: Apply optimistic locking to entities where concurrent modification is a realistic concern and data integrity is critical. Not every table or resource needs it. Overusing it can add unnecessary complexity.
Always Reload on Conflict: Before retrying an update after a conflict, always reload the record. This ensures you have the latest version number and the most current state of other attributes, preventing further issues or lost updates.
Combine with Other Techniques (If Needed): For highly contended resources where optimistic lock failures become too frequent, you might need to explore other strategies for those specific parts of your application, such as more granular locking, CQRS patterns, or even considering pessimistic locking as a last resort if the cost of retries is too high.
Choose the Right Version Column Type: Typically, an INTEGER
or BIGINT
(Int32
or Int64
in Crystal) is suitable. Ensure it won't realistically overflow during the lifetime of a record.
Atomic Increment by Database: Rely on the database's atomicity for the UPDATE ... SET version = version + 1 WHERE version = old_version
pattern. CQL's implementation handles this.
Thorough Testing: Write specific tests for conflict scenarios to ensure your conflict resolution logic works as expected. Simulate concurrent updates in your tests.
Performance Monitoring: Monitor the frequency of OptimisticLockError
s. A consistently high rate might indicate that the "optimistic" assumption (conflicts are rare) is not holding for that particular resource, and a different concurrency strategy might be more efficient.
Optimistic locking, as implemented with a version column incremented by the application (via the ORM), is a widely portable pattern that works consistently across most relational databases. The core mechanism relies on standard SQL features (UPDATE
with a WHERE
clause checking the version and SET
to increment it).
Here's how it generally applies to common databases used with Crystal:
Behavior: Works seamlessly. PostgreSQL's robust MVCC (Multi-Version Concurrency Control) architecture is well-suited for optimistic locking.
Version Column Type: INTEGER
or BIGINT
are excellent choices.
Transactions: PostgreSQL has strong ACID-compliant transactions. Use them to group related operations where atomicity is required, in conjunction with optimistic locking for individual record updates.
No Special Configuration: No special database-level configuration is typically needed for CQL's optimistic locking to function correctly.
Behavior: Works well, especially with the InnoDB storage engine (which is the default for modern MySQL versions and provides row-level locking and transactions).
Version Column Type: INT
or BIGINT
are appropriate.
Transactions: InnoDB supports ACID transactions. As with PostgreSQL, use transactions for atomicity of larger operations.
Isolation Levels: While optimistic locking itself relies on the version check, ensure your transaction isolation level (e.g., REPEATABLE READ
or READ COMMITTED
for InnoDB) meets your application's general consistency requirements. Optimistic locking can help reduce the need for stricter isolation levels for the sole purpose of preventing lost updates.
Behavior: Optimistic locking is fully functional and effective with SQLite.
Version Column Type: INTEGER
is standard.
Concurrency Model: SQLite, by default, serializes writes at the database level (multiple connections can read concurrently, but writes are typically queued). However, if your application has multiple threads or processes accessing the same connection without proper external synchronization, or if multiple independent processes access the same database file, optimistic locking is still crucial to prevent lost updates.
WAL Mode: Using Write-Ahead Logging (WAL) mode in SQLite can improve concurrency for readers and a single writer. Optimistic locking remains a valuable pattern even with WAL mode to manage conflicts if multiple writers could conceptually attempt updates based on stale data.
Simplicity: For simpler applications or embedded use cases where SQLite shines, optimistic locking provides a straightforward way to handle potential concurrency issues without complex locking mechanisms.
In summary: CQL's optimistic locking implementation is designed to be database-agnostic by relying on standard SQL patterns. The primary responsibility of the database is to atomically execute the conditional UPDATE
statement. The choice of database mainly influences broader aspects like overall transaction management, concurrency capabilities, and performance characteristics, but the optimistic locking logic itself remains consistent.
Increased Retries: Not suitable for high-contention resources where conflicts would occur very frequently, as this leads to many transaction rollbacks and retries, potentially degrading performance.
Application-Level Handling: Requires application logic to catch the OptimisticLockError
and implement a retry or conflict resolution strategy. It's not transparent like some pessimistic locking mechanisms.
Slight Write Overhead: Each update involves checking and incrementing the version column, which is a minimal but present overhead compared to an update without version checking.
"Last Commit Wins" (Effectively): While it prevents lost updates, the record will reflect the state of the last successful commit. If two users make conflicting changes, the first one to commit "wins" for that version, and the second must handle the conflict.
Migrations in CQL provide a structured way to evolve your database schema over time. They allow you to create, modify, and drop tables and columns in a versioned, reversible manner using Crystal code.
A migration is a Crystal class that defines changes to your database schema. Each migration has a unique version number and two methods:
up
: Applies the migration (e.g., creates or alters tables).
down
: Reverts the migration (e.g., drops or reverts changes).
Create a new migration by subclassing CQL::Migration
and setting a unique version:
class CreateUsers < CQL::Migration
self.version = 1_i64
def up
schema.create :users do
primary_key :id, :serial
column :name, :string
column :email, :string
column :active, :bool, default: false
column :created_at, :timestamp
column :updated_at, :timestamp
end
end
def down
schema.drop :users
end
end
self.version
must be unique and increasing (often a timestamp or integer).
The up
method defines the schema changes to apply.
The down
method defines how to revert those changes.
You can run migrations using your preferred migration runner or a custom script. A typical workflow:
Place migration files in a db/migrations/
directory.
Load and apply migrations in order:
require "cql"
# Load all migration files
dir = "./db/migrations"
Dir.glob("#{dir}/*.cr").each { |file| require file }
# Run all migrations (pseudo-code, depends on your runner)
CQL::Migrator.new(AcmeDB).migrate!
Consult your project's migration runner or CQL's documentation for details on migration management.
You can alter existing tables in a migration:
class AddAgeToUsers < CQL::Migration
self.version = 2_i64
def up
schema.alter :users do
add_column :age, :int32
end
end
def down
schema.alter :users do
drop_column :age
end
end
end
Use a unique, increasing version for each migration.
Write reversible migrations (always provide a down
method).
Keep migrations in version control.
Test migrations on a development database before running in production.
Use descriptive class names (e.g., AddAgeToUsers
).
class CreatePosts < CQL::Migration
self.version = 3_i64
def up
schema.create :posts do
primary_key :id, :serial
column :user_id, :int32
column :title, :string
column :body, :string
column :created_at, :timestamp
end
end
def down
schema.drop :posts
end
end
class AddIndexToUsersEmail < CQL::Migration
self.version = 4_i64
def up
schema.alter :users do
add_index :email, unique: true
end
end
def down
schema.alter :users do
drop_index :email
end
end
end
For more on defining models and querying, see the other guides in this directory.
CQL's Active Record integration provides a simple yet effective way to ensure data integrity through model-level validations. By defining validation logic within your models, you can prevent invalid data from being persisted to the database.
The CQL::ActiveRecord::Validations
module is automatically included when you use CQL::ActiveRecord::Model(Pk)
.
Validations are typically implemented by overriding the validate
instance method in your model. This method is called automatically before saving a record (create or update).
Inside the validate
method, you should check the model's attributes. If an attribute is invalid, you add a message to the errors
collection for that attribute.
Model Example:
struct User
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :users
property id : Int64?
property name : String
property email : String
property age : Int32?
def initialize(@name : String, @email : String, @age : Int32? = nil)
end
# Override the validate method to add custom validation logic
def validate
super # It's good practice to call super in case parent classes have validations
if name.empty?
errors.add(:name, "cannot be blank")
end
if email.empty?
errors.add(:email, "cannot be blank")
elsif !email.includes?('@')
errors.add(:email, "is not a valid email format")
end
if age_val = age
if age_val < 18
errors.add(:age, "must be 18 or older")
end
end
end
end
You can explicitly check if a model instance is valid using the valid?
and validate!
methods.
valid?
The valid?
method runs the validate
method and returns true
if the errors
collection is empty, and false
otherwise. It does not raise an exception.
user = User.new(name: "", email: "test")
user.valid? # => false
puts user.errors.full_messages.join(", ") # => "Name cannot be blank, Email is not a valid email format"
valid_user = User.new(name: "Jane Doe", email: "jane@example.com", age: 30)
valid_user.valid? # => true
puts valid_user.errors.empty? # => true
validate!
The validate!
method also runs the validate
method. If the record is invalid, it raises a CQL::Errors::RecordInvalid
exception containing the validation errors. If the record is valid, it returns true
.
user = User.new(name: "", email: "test")
begin
user.validate!
rescue ex : CQL::Errors::RecordInvalid
puts ex.message # Displays the summary of errors
puts ex.record.errors.full_messages.join(", ") # => "Name cannot be blank, Email is not a valid email format"
end
valid_user = User.new(name: "John Doe", email: "john@example.com", age: 25)
valid_user.validate! # => true (no exception raised)
errors
ObjectThe errors
object is an instance of CQL::Errors::Collection
. It provides methods to add and inspect error messages.
errors.add(attribute : Symbol, message : String)
: Adds an error message for the specified attribute.
errors.empty?
: Returns true
if there are no errors.
errors.clear
: Removes all error messages.
errors.on(attribute : Symbol)
: Returns an array of error messages for a specific attribute, or nil
if none.
errors.full_messages
: Returns an array of user-friendly error messages, typically in the format "Attribute_name message" (e.g., "Name cannot be blank").
errors.to_h
: Returns a hash where keys are attribute names (Symbols) and values are arrays of error messages for that attribute.
user = User.new(name: "", email: "bademail", age: 10)
user.valid? # Run validations to populate errors
puts user.errors.on(:name) # => ["cannot be blank"]
puts user.errors.on(:email) # => ["is not a valid email format"]
puts user.errors.on(:age) # => ["must be 18 or older"]
puts user.errors.on(:non_existent) # => nil
puts user.errors.full_messages
# [
# "Name cannot be blank",
# "Email is not a valid email format",
# "Age must be 18 or older"
# ]
pp user.errors.to_h
# {
# name: ["cannot be blank"],
# email: ["is not a valid email format"],
# age: ["must be 18 or older"]
# }
Validation is automatically integrated with the persistence methods:
save
/ save!
:
save
runs validations. If they fail (valid?
returns false
), save
returns false
and the record is not persisted.
save!
also runs validations. If they fail, it raises CQL::Errors::RecordInvalid
.
create
/ create!
:
create(attributes)
is like new(attributes).save
. It returns the instance (which may be invalid and not persisted) or false
if using a before_create
callback that halts.
create!(attributes)
is like new(attributes).save!
. It raises CQL::Errors::RecordInvalid
if validations fail.
update(attributes)
/ update!(attributes)
:
update
assigns attributes and then calls save
. Returns true
or false
.
update!
assigns attributes and then calls save!
. Raises CQL::Errors::RecordInvalid
on failure.
Example with save
:
invalid_user = User.new(name: "", email: "test")
if invalid_user.save
puts "User saved successfully!" # This won't be printed
else
puts "Failed to save user."
puts invalid_user.errors.full_messages.join(", ")
# => Failed to save user.
# => Name cannot be blank, Email is not a valid email format
end
valid_user = User.new(name: "Alice", email: "alice@example.com", age: 30)
if valid_user.save
puts "User '#{valid_user.name}' saved successfully! ID: #{valid_user.id.not_nil!}"
else
puts "Failed to save valid user." # This won't be printed
end
Example with create!
:
begin
User.create!(name: "", email: "invalid")
rescue ex : CQL::Errors::RecordInvalid
puts "Caught error on create!: #{ex.message}"
end
new_user = User.create!(name: "Bob", email: "bob@example.com", age: 40)
puts "User '#{new_user.name}' created successfully! ID: #{new_user.id.not_nil!}"
For more complex or reusable validation logic, you can define private helper methods within your model.
struct Product
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :products
property id : Int64?
property name : String
property price : Float64
property sku : String
def initialize(@name : String, @price : Float64, @sku : String)
end
def validate
super
validate_name
validate_price
validate_sku_format
end
private def validate_name
errors.add(:name, "cannot be empty") if name.empty?
end
private def validate_price
errors.add(:price, "must be positive") if price <= 0
end
# Example: SKU must be in format ABC-12345
private def validate_sku_format
unless sku.matches?(/^[A-Z]{3}-\d{5}$/)
errors.add(:sku, "format is invalid (expected ABC-12345)")
end
end
end
# Example usage:
product = Product.new(name: "Test Product", price: -10.0, sku: "XYZ-123")
product.valid? # => false
pp product.errors.to_h
# {
# price: ["must be positive"],
# sku: ["format is invalid (expected ABC-12345)"]
# }
By using these validation features, you can maintain data consistency and provide clear feedback about data entry issues.
Remember that CQL::ActiveRecord::Validations
provides a basic framework. For very complex validation scenarios or integrating with external validation libraries, you might need to extend this functionality further.
CQL's Active Record provides lifecycle callbacks that allow you to trigger logic at various points during an object's life. This is useful for tasks like data normalization before validation, encrypting passwords before saving, or sending notifications after an action.
The CQL::ActiveRecord::Callbacks
module is automatically included when you use CQL::ActiveRecord::Model(Pk)
.
Callbacks are methods defined in your model that are registered to be called at specific moments. CQL supports the following callbacks:
Validation Callbacks:
before_validation(method_name)
: Called before validate
is run.
after_validation(method_name)
: Called after validate
completes.
Save Callbacks (run for both create and update):
before_save(method_name)
: Called before the record is saved to the database.
after_save(method_name)
: Called after the record is saved to the database.
Create Callbacks (run only when a new record is saved):
before_create(method_name)
: Called before a new record is inserted into the database.
after_create(method_name)
: Called after a new record is inserted into the database.
Update Callbacks (run only when an existing record is saved):
before_update(method_name)
: Called before an existing record is updated in the database.
after_update(method_name)
: Called after an existing record is updated in the database.
Destroy Callbacks:
before_destroy(method_name)
: Called before a record is deleted from the database.
after_destroy(method_name)
: Called after a record is deleted from the database.
You register a callback by calling its macro with the name of the method (as a Symbol) to be executed.
In this example:
normalize_email
will run before validations.
set_status_if_nil
will run before any save operation (create or update).
send_welcome_email
will run only after a new user is created.
record_login_time
will run only before an existing user is updated.
log_deletion
will run after a user is destroyed.
If a before_validation
, before_save
, before_create
, before_update
, or before_destroy
callback method returns false
(explicitly false
, not nil
or other falsy values), the callback chain is halted. This means:
Subsequent callbacks of the same type (e.g., other before_save
methods) will not be executed.
The main action (validation, save, create, update, or destroy) will be canceled.
For save
, create
, update
, it will return false
.
For save!
, create!
, update!
, it will not raise RecordInvalid
due to validation errors (if any ran), but simply won't proceed with persistence.
For destroy
, it will return false
, and the record will not be deleted.
Example of halting:
after_*
callbacks do not have the power to halt the chain, as the primary action has already completed.
When multiple callbacks are registered for the same event, they are executed in the order they were defined in the model.
CQL aims to follow a similar callback order to other popular ORMs like Rails Active Record during the save
process:
before_validation
Validations are run (validate
method)
after_validation
before_save
If new record: before_create
If existing record: before_update
Database operation (INSERT or UPDATE)
If new record: after_create
If existing record: after_update
after_save
For destroy
:
before_destroy
Database operation (DELETE)
after_destroy
Data Manipulation: Normalize data (e.g., downcasing emails), set default values, generate tokens.
Lifecycle Management: Update related objects, log changes, manage state transitions.
Notifications: Send emails or push notifications after certain events (e.g., after_create
).
Conditional Logic: A callback method can contain logic to decide if it should perform an action, or even halt the entire operation.
Callbacks are a powerful tool for adding behavior to your models without cluttering your controller or service logic. However, use them judiciously, as complex callback chains can sometimes make debugging harder.
struct User
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :users
property id : Int64?
property name : String
property email : String
property status : String?
property last_logged_in_at : Time?
def initialize(@name : String, @email : String)
@status = "pending"
end
# Registering callbacks
before_validation :normalize_email
before_save :set_status_if_nil
after_create :send_welcome_email
before_update :record_login_time
after_destroy :log_deletion
private def normalize_email
self.email = email.downcase.strip
true # Important: before_ callbacks must return true or a truthy value to continue
end
private def set_status_if_nil
self.status = "active" if status.nil?
true
end
private def send_welcome_email
puts "Sending welcome email to #{email}..."
# In a real app, you'd use an email library here
true
end
private def record_login_time
# Hypothetical: this callback might be triggered indirectly
# if an update operation is intended to signify a login.
# More commonly, you'd set this explicitly.
self.last_logged_in_at = Time.utc
true
end
private def log_deletion
puts "User #{name} (ID: #{id.inspect}) deleted."
true # after_ callbacks don't halt the chain, but consistency is good
end
end
struct Article
include CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :articles
property id : Int64?
property title : String
property published : Bool
def initialize(@title : String, @published : Bool = false)
end
before_save :check_if_can_be_published
private def check_if_can_be_published
if published && title.empty?
errors.add(:title, "cannot be blank if published")
return false # Halt the save
end
true
end
end
# Attempt to save an article that will be halted
article1 = Article.new(title: "", published: true)
if article1.save
puts "Article 1 saved!" # This won't run
else
puts "Article 1 not saved. Errors: #{article1.errors.full_messages.join(", ")}"
# Output: Article 1 not saved. Errors: Title cannot be blank if published
end
# Attempt to save a valid article
article2 = Article.new(title: "My Great Article", published: true)
if article2.save
puts "Article 2 saved! ID: #{article2.id.inspect}"
else
puts "Article 2 not saved."
end
This section provides comprehensive guides for defining and working with associations between models in CQL Active Record. Relationships are a core part of modeling real-world data and enable you to express connections such as ownership, membership, and linkage between records.
Overview of Association Types: Learn the differences between one-to-one, one-to-many, and many-to-many relationships.
Step-by-Step Guides: Each sub-guide covers how to set up, use, and query each type of association in CQL, with practical code examples.
Best Practices: Tips for managing associations, avoiding common pitfalls, and writing maintainable code.
Belongs To: How to set up and use belongs_to
associations (e.g., a comment belongs to a post).
Has One: How to define and work with has_one
relationships (e.g., a user has one profile).
Has Many: How to manage has_many
associations (e.g., a post has many comments).
Many To Many: How to implement many-to-many relationships using join tables (e.g., posts and tags).
Each guide includes:
Schema and model setup
Creating and querying associations
Modifying and deleting related records
Eager loading and performance tips
Use these guides to master associations in your CQL-powered Crystal applications!
ManyToMany
Relationship GuideIn this guide, we'll cover the ManyToMany
relationship using CQL's Active Record syntax. This relationship is used when multiple records in one table can relate to multiple records in another table, facilitated by an intermediate join table.
ManyToMany
Relationship?A ManyToMany
relationship means that multiple records in one table can relate to multiple records in another table. For example:
A Post can have multiple Tags (e.g., "Tech", "News").
A Tag can belong to multiple Posts (e.g., both Post 1 and Post 2 can have the "Tech" tag).
We'll use a scenario where:
A Post can have many Tags.
A Tag can belong to many Posts.
We will represent this many-to-many relationship using a join table called PostTags.
We'll define the posts
, tags
, and post_tags
tables in the schema using CQL's DSL.
posts table: Stores post details such as title
, body
, and published_at
.
tags table: Stores tag names.
post_tags table: A join table that connects posts
and tags
via their foreign keys post_id
and tag_id
.
Let's define the Post
, Tag
, and PostTag
models in CQL, establishing the ManyToMany
relationship.
The many_to_many :tags, Tag, join_through: :post_tags
association in the Post
model connects Post
to Tag
via the post_tags
table.
Similarly, the Tag
model uses many_to_many :posts, Post, join_through: :post_tags
.
The PostTag
model is crucial. It belongs_to
both Post
and Tag
.
The many_to_many
macro uses this join model implicitly via the join_through: :post_tags
option, which refers to the table name.
ManyToMany
AssociationsWhen you access a many_to_many
association (e.g., post.tags
), you get a ManyCollection
proxy that offers powerful methods to manage the relationship.
Removing an association (deletes the join table record, not the Tag itself):
Clearing all associations for a post (deletes all its PostTag records):
clear
only removes the join records. The Tag
records themselves are not deleted.
If cascade: true
was set on the many_to_many
association, the behavior of delete
or clear
with respect to the target records (Tag
) might change, potentially deleting them. This needs careful checking of ManyCollection
's cascade implementation.
Deleting a Post
or a Tag
will not automatically delete its associations from the post_tags
table or the associated records on the other side, unless cascade deletes are configured at the database level or handled by before_destroy
callbacks manually cleaning up join table records.
ManyToMany
To avoid N+1 queries with many-to-many associations, use includes
:
In this guide, we explored the ManyToMany
relationship in CQL:
Defined Post
, Tag
, and PostTag
(join) models using CQL::ActiveRecord::Model(Pk)
.
Used the many_to_many :association, TargetModel, join_through: :join_table_symbol
macro.
Showcased managing associations using ManyCollection
methods like <<
, create
, delete
, and clear
.
Discussed accessing associated records and eager loading with includes
.
This provides a robust way to handle many-to-many relationships in your Crystal applications using CQL.
This concludes our series of guides on relationships in CQL Active Record, covering BelongsTo
, HasOne
, HasMany
, and ManyToMany
. Feel free to experiment by extending your models, adding validations, or implementing more complex queries to suit your needs.
CQL provides a powerful, type-safe query builder that allows you to compose sophisticated SQL queries using idiomatic Crystal code. This guide demonstrates how to build complex queries using Model.query
and the CQL::Query
interface.
Combine multiple conditions, logical operators, and expressions:
Join multiple tables, use aliases, and filter on joined data:
Use grouping and aggregate functions for reporting and analytics:
Use subqueries for advanced filtering and data retrieval:
Use CTEs for reusable subqueries and complex data transformations:
Combine pagination with other query methods:
Use table aliases for clarity in multi-join queries.
Use block syntax for complex conditions to improve readability.
Use .select
to limit columns and improve performance.
Use .group_by
and aggregates for reporting.
Use subqueries and CTEs for advanced analytics and filtering.
Always test and inspect generated SQL with .to_sql
for correctness and performance.
For more details on the query interface, see the .
AcmeDB = CQL::Schema.define(
:acme_db,
adapter: CQL::Adapter::Postgres,
uri: ENV["DATABASE_URL"]
) do
table :posts do
primary # Defaults to :id, Int64, auto_increment: true
text :title
text :body
timestamp :published_at
end
table :tags do
primary
text :name
end
table :post_tags do
primary
bigint :post_id, index: true
bigint :tag_id, index: true
index [:post_id, :tag_id], unique: true
end
end
struct Post < CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :posts
property id : Int64?
property title : String
property body : String
property published_at : Time?
# Initializing a new post
def initialize(@title : String, @body : String, @published_at : Time? = Time.utc)
end
# Association: A Post has many Tags through the 'post_tags' join table.
# `join_through` refers to the table name of the join model.
many_to_many :tags, Tag, join_through: :post_tags
end
struct Tag < CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :tags
property id : Int64?
property name : String
# Initializing a new tag
def initialize(@name : String)
end
# Association: A Tag has many Posts through the 'post_tags' join table.
many_to_many :posts, Post, join_through: :post_tags
end
struct PostTag < CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :post_tags
property id : Int64?
property post_id : Int64?
property tag_id : Int64?
# Initializing a PostTag. IDs are typically set by the association logic.
def initialize(@post_id : Int64? = nil, @tag_id : Int64? = nil)
end
# Associations to the parent models
belongs_to :post, Post, foreign_key: :post_id
belongs_to :tag, Tag, foreign_key: :tag_id
end
# Create a new Post
post = Post.create!(title: "Crystal Language Guide", body: "This is a guide about Crystal.")
# Option 1: Create new Tags and associate them via the collection's `create` method
# This creates the Tag, saves it, and creates the PostTag join record.
tag_crystal = post.tags.create(name: "Crystal")
tag_programming = post.tags.create(name: "Programming")
puts "Post '#{post.title}' now has tags: #{post.tags.all.map(&.name).join(", ")}"
# Option 2: Add existing (persisted) Tags using `<<`
# First, ensure the tags exist or create them.
tag_tech = Tag.find_or_create_by(name: "Tech")
tag_tech.save! if !tag_tech.persisted?
post.tags << tag_tech
puts "After adding 'Tech', Post tags: #{post.tags.all.map(&.name).join(", ")}"
# Trying to add an unpersisted tag with `<<` will raise an error.
# new_unpersisted_tag = Tag.new(name: "Unsaved")
# begin
# post.tags << new_unpersisted_tag # This would fail
# rescue e : ArgumentError
# puts "Error adding unsaved tag: #{e.message}"
# end
# Option 3: Setting associations by an array of IDs (post.tag_ids = [...])
# This is a common pattern for updating all associations at once.
# It typically clears existing associations for this post and creates new ones.
# Ensure some tags exist to get their IDs
tag_guide = Tag.find_or_create_by(name: "Guide") { |t| t.save! }
tag_cql = Tag.find_or_create_by(name: "CQL") { |t| t.save! }
# Assuming ManyCollection supports `ids=` (verify from ManyCollection API)
# post.tag_ids = [tag_crystal.id.not_nil!, tag_guide.id.not_nil!, tag_cql.id.not_nil!]
# This functionality (direct assignment to `tag_ids=`) depends on its specific implementation in `ManyCollection`.
# If not directly available, an alternative is to clear and then add:
post.tags.clear
post.tags << tag_crystal
post.tags << tag_guide
post.tags << tag_cql
puts "After setting by IDs (clear & add), Post tags: #{post.tags.all.map(&.name).join(", ")}"
# Fetch the post
loaded_post = Post.find!(post.id.not_nil!)
puts "\nTags for post '#{loaded_post.title}':"
loaded_post.tags.all.each do |tag|
puts "- #{tag.name}"
end
# Fetch the tag and its posts
loaded_tag_crystal = Tag.find!(tag_crystal.id.not_nil!)
puts "\nPosts for tag '#{loaded_tag_crystal.name}':"
loaded_tag_crystal.posts.all.each do |p|
puts "- #{p.title}"
end
# Other collection methods like find_by, exists?, size, empty? also work:
tech_tag_on_post = loaded_post.tags.find_by(name: "Tech")
puts "Post has 'Tech' tag? #{!tech_tag_on_post.nil?}"
# Remove the 'Programming' tag association from the post
programming_tag = loaded_post.tags.find_by(name: "Programming")
if prog_tag = programming_tag
if loaded_post.tags.delete(prog_tag) # Pass instance or its ID
puts "Removed 'Programming' tag association from '#{loaded_post.title}'."
end
end
puts "Post tags after removing 'Programming': #{loaded_post.tags.all.map(&.name).join(", ")}"
# The 'Programming' tag itself still exists in the `tags` table
still_exists_programming_tag = Tag.find_by(name: "Programming")
puts "'Programming' tag still exists globally? #{!still_exists_programming_tag.nil?}"
puts "Tags before clear for '#{loaded_post.title}': #{loaded_post.tags.size}"
loaded_post.tags.clear
puts "Tags after clear for '#{loaded_post.title}': #{loaded_post.tags.size}" # Should be 0
# Fetches all posts and their associated tags efficiently
posts_with_tags = Post.query.includes(:tags).all(Post)
posts_with_tags.each do |p|
puts "Post: #{p.title} has tags: #{p.tags.map(&.name).join(", ") || "None"} (already loaded)"
end
# Multiple AND/OR conditions
users = User.query.where { (age >= 18) & (active == true) }
.where { (name.like("A%")) | (email.like("%@gmail.com")) }
.all(User)
# Nested conditions
admins = User.query.where { (role == "admin") & ((status == "active") | (status == "pending")) }
.all(User)
# IN and NOT IN
moderators = User.query.where { role.in(["admin", "moderator"]) }
.all(User)
excluded = User.query.where { id.not_in([1, 2, 3]) }
.all(User)
# Join with conditions
users_with_posts = User.query.join(Post) { users.id == posts.user_id }
.where { posts.published == true }
.all(User)
# Multiple joins with aliases
User.query.from(users: :u)
.join({posts: :p}) { u.id == p.user_id }
.join({comments: :c}) { p.id == c.post_id }
.where { c.created_at > 1.week.ago }
.all(User)
# Group by and aggregate
role_stats = User.query.select { [role, CQL.count(id).as("total")] }
.group_by(:role)
.all(User)
# Group by multiple columns
stats = Post.query.select { [user_id, CQL.count(id).as("post_count")] }
.group_by(:user_id)
.having { CQL.count(id) > 5 }
.all(Post)
# Aggregates: sum, avg, min, max
summary = Order.query.select { [CQL.sum(:total).as("total_sum"), CQL.avg(:total).as("avg_total")] }
.where { status == "paid" }
.all(Order)
# Users with more than 5 posts
users = User.query.where { id.in(
Post.query.select(:user_id)
.group_by(:user_id)
.having { CQL.count(id) > 5 }
) }.all(User)
# Users with recent activity in posts or comments
users = User.query.where { id.in(
Post.query.select(:user_id)
.where { created_at > 1.week.ago }
.union(
Comment.query.select(:user_id)
.where { created_at > 1.week.ago }
)
) }.all(User)
# Users with their post and comment counts
User.query.with(:user_stats) {
User.query.select { [id,
CQL.count(posts.id).as("post_count"),
CQL.count(comments.id).as("comment_count")] }
.left_join(Post) { users.id == posts.user_id }
.left_join(Comment) { users.id == comments.user_id }
.group_by(:id)
}
.select { [users.*, user_stats.*] }
.join(:user_stats) { users.id == user_stats.id }
.all(User)
# Paginated search with sorting
page = 2
per_page = 20
users = User.query.where { name.like("%search%") }
.order(:name)
.offset((page - 1) * per_page)
.limit(per_page)
.all(User)
Database migrations are a crucial part of managing your application's database schema in a structured and version-controlled way. CQL provides a migration system that allows you to evolve your database schema over time as your application's requirements change.
Migrations are Crystal classes that define how to apply changes (up
method) and how to revert them (down
method).
As your application evolves, you'll often need to:
Create new tables.
Add, remove, or modify columns in existing tables.
Add or remove indexes.
Perform other schema alterations.
Migrations allow you to script these changes in Crystal code. Each migration file typically represents a single, atomic change to the database schema and is often timestamped or versioned to ensure changes are applied in the correct order.
Benefits of using migrations:
Version Control: Schema changes are tracked in your project's version control system (e.g., Git) alongside your application code.
Collaboration: Makes it easier for teams to manage database schema changes consistently across different development environments.
Reproducibility: Ensures that the database schema can be recreated reliably in any environment (development, testing, production).
Automation: Schema changes can be applied automatically as part of deployment processes.
Migrations in CQL are Crystal classes that inherit from CQL::Migration
(or a similar base class provided by the CQL framework).
File Naming and Location: Migration files are typically placed in a db/migrate/
directory within your project. The filename often includes a timestamp or a sequential version number to denote the order of execution, followed by a descriptive name for the migration (e.g., db/migrate/20231027000000_create_users.cr
).
Migration Class Structure:
# db/migrate/YYYYMMDDHHMMSS_create_users.cr
# Replace YYYYMMDDHHMMSS with the actual timestamp for the migration.
class CreateUsers < CQL::Migration
# Optional: Define a version for this migration if not derived from filename.
# self.version = 20231027000000_i64 # Example version
# The `up` method describes the changes to apply to the database.
def up(schema : CQL::Schema::Definition)
# Example: Create a 'users' table
schema.create_table :users do |t|
# Define columns for the table
t.primary_key :id # Defines an auto-incrementing primary key named 'id'.
# Specifics like `bigserial` for PostgreSQL might be abstracted or configurable.
t.text :name, null: false # A text column for user's name, cannot be null.
t.text :email, null: false, unique: true # Email, cannot be null, must be unique.
t.bool :active, default: false # Boolean for active status, defaults to false.
t.timestamps # Convenience method to add `created_at` and `updated_at` (both Time, nullable by default usually).
end
# Example: Adding an index separately (if not done in create_table)
schema.add_index :users, :email, unique: true, name: "index_users_on_email_unique"
end
# The `down` method describes how to revert the changes made in the `up` method.
def down(schema : CQL::Schema::Definition)
# Example: Drop the 'users' table
schema.drop_table :users
# If you added an index separately in `up`, you might remove it here:
# schema.remove_index :users, name: "index_users_on_email_unique" (or by column)
end
end
Key Points:
class CreateUsers < CQL::Migration
: Your migration class inherits from CQL::Migration
.
up(schema : CQL::Schema::Definition)
: This method is called when applying the migration. The schema
object provides methods to manipulate the database structure.
down(schema : CQL::Schema::Definition)
: This method is called when reverting (rolling back) the migration. It should undo the changes made by the up
method.
Schema Definition API: The methods available on the schema
object (e.g., create_table
, add_column
, drop_table
, add_index
) and the column definition block (e.g., t.text
, t.bool
, t.primary_key
, t.timestamps
) are specific to CQL. The exact syntax and available types (:text
, :integer
, :bool
, :datetime
, etc.) should be referenced from the CQL documentation you are using. The example above uses common conventions.
Here are some common operations you might perform within the up
and down
methods of a migration:
Create Table (schema.create_table :table_name do |t| ... end
):
# up method
schema.create_table :posts do |t|
t.primary_key :id
t.references :user, foreign_key: true # Creates user_id and a foreign key constraint
t.text :title, null: false
t.text :body
t.datetime :published_at
t.timestamps
end
# down method
schema.drop_table :posts
Drop Table (schema.drop_table :table_name
):
Used in the down
method to remove a table created in the up
method.
Alter Table (schema.alter_table :table_name do |t| ... end
):
Used for adding, removing, or changing columns on an existing table.
create_table
or alter_table
)Add Column (t.add_column :column_name, :type, options...
or schema.add_column :table, :column, :type, ...
):
# up method (inside alter_table)
# schema.alter_table :users do |t|
# t.integer :login_count, default: 0
# end
# Or, if alter_table is not the direct way for add_column with CQL
schema.add_column :users, :login_count, :integer, default: 0, null: false
# down method
schema.remove_column :users, :login_count
Remove Column (t.remove_column :column_name
or schema.remove_column :table, :column
):
Used to drop a column.
Rename Column (schema.rename_column :table, :old_name, :new_name
):
Changes the name of an existing column.
Change Column (schema.change_column :table, :column, :new_type, options...
):
Modifies the type or other options (like null
, default
) of an existing column.
# up method
# schema.change_column :users, :email, :text, limit: 255 # Example if changing type or options
Add Index (schema.add_index :table_name, :column_name_or_columns, options...
):
Indexes improve query performance on frequently searched columns.
# up method
schema.add_index :posts, :user_id
schema.add_index :posts, [:title, :published_at], name: "idx_posts_on_title_and_published_at"
Remove Index (schema.remove_index :table_name, :column_name_or_columns
or name: :index_name
):
# down method
schema.remove_index :posts, :user_id
schema.remove_index :posts, name: "idx_posts_on_title_and_published_at"
Foreign keys can often be defined when creating columns (e.g., t.references :author, foreign_key: true
) or added separately using methods like schema.add_foreign_key
.
# Assuming t.references in create_table does this.
# If not, explicitly:
schema.add_foreign_key :posts, :users, column: :user_id, primary_key: :id
Important: The exact method names and options for schema manipulation (create_table
, add_column
, available data types like :text
, :integer
, :bool
, options like null:
, default:
, unique:
) can vary significantly between different database adapters (PostgreSQL, MySQL) and ORM/query builder implementations. Always refer to the specific CQL documentation for the version you are using to ensure you are using the correct API for schema definition.
CQL will typically provide command-line tools or Rake tasks (if integrated with Rake) to manage and run your migrations.
Common migration commands (exact syntax will depend on CQL's tooling):
db:migrate
: Applies all pending migrations (those that haven't been run yet).
# Example placeholder command - replace with actual CQL command
# crystal run path/to/cql/runner.cr db:migrate CONTEXT=YourDBContext
# or if using Rake:
# rake db:migrate
db:rollback
: Reverts the last applied migration.
# rake db:rollback
db:schema:load
: Loads the schema from a schema file (e.g., db/schema.cr
or db/structure.sql
) into the database. This is often used to set up a new database quickly by loading the current schema state, bypassing running all migrations individually.
db:schema:dump
: Creates or updates a schema file based on the current state of the database. This file represents the authoritative structure of your database.
db:reset
: Typically drops the database, recreates it, and then loads the schema (or runs all migrations). Useful for resetting the database to a clean state in development.
Checking Migration Status: Tools to see which migrations have been applied and which are pending.
Database Context: When running migrations, you often need to specify the database context (e.g., CONTEXT=AcmeDB
) if your application uses multiple databases or if the migration runner needs to know which configuration to use.
db/schema.cr
or db/structure.sql
)After migrations are run, CQL (like many ORMs) may maintain a db/schema.cr
or db/structure.sql
file.
db/schema.cr
(if applicable): This would be a Crystal representation of your current database schema, generated by inspecting the database. It's often used by db:schema:load
.
db/structure.sql
: Alternatively, a raw SQL dump of the database structure. This is database-agnostic for loading but less so for inspection.
This file serves as the canonical representation of your database schema at a given point in time. It's recommended to commit this file to version control.
Migrations are a powerful tool for database schema management. Writing reversible migrations (up
and down
methods) is crucial, especially for rolling back changes if needed.
Always test your migrations thoroughly, especially those involving data transformation or potentially destructive operations.
Query scopes in CQL Active Record allow you to define reusable query constraints on your models. They help in making your code D.R.Y. (Don't Repeat Yourself) by encapsulating common query logic, leading to more readable and maintainable model and controller code.
The primary way to define scopes in CQL is by using the scope
macro. Alternatively, scopes can also be defined as class methods.
Often, you'll find yourself writing the same query conditions repeatedly. For example:
Fetching all published articles.
Finding all active users.
Retrieving items created in the last 7 days.
Scopes let you give names to these common queries. A scope is essentially a pre-defined query or a piece of a query that can be easily applied and chained.
Benefits of using scopes:
Readability: Article.published.all
is much clearer than Article.query.where(status: "published").order(published_at: :desc).all(Article)
scattered throughout your codebase.
Reusability: Define the logic once and use it anywhere you need that specific dataset.
Maintainability: If the definition of "published" changes, you only need to update it in one place (the scope definition).
Chainability: Scopes can be chained with other scopes or standard query methods.
There are two main ways to define scopes in CQL: using the scope
macro (recommended) or defining them as class methods.
scope
Macro (Recommended)The most concise and recommended way to define scopes is by using the scope
macro. This macro is provided by the CQL::ActiveRecord::Scopes
module, which you should include in your model.
Syntax:
scope :scope_name, ->(optional_arguments) { query_logic }
:scope_name
: The name of the scope (a Symbol
), which will become a class method on your model.
->(optional_arguments) { ... }
: A proc that defines the query logic.
It can optionally take arguments with their types, e.g., ->(count : Int32, category : String) { ... }
.
Inside the proc, you use standard CQL query methods like where
, order
, limit
, etc. These methods are called on the current query context, which is typically a ChainableQuery(YourModel)
instance or the model class itself if starting a new chain.
The proc should return a CQL::Query
or a ChainableQuery(YourModel)
. If a raw CQL::Query
is returned (e.g., by starting with query.where(...)
), the scope
macro intelligently wraps it in a ChainableQuery(YourModel)
to ensure it remains chainable with other scopes or Active Record query methods.
Examples:
struct Article
includes CQL::ActiveRecord::Model(Int64)
includes CQL::ActiveRecord::Scopes # Important: Include the Scopes module
db_context AcmeDB, :articles
property id : Int64?
property title : String
property status : String # e.g., "draft", "published", "archived"
property view_count : Int32 = 0
property published_at : Time?
# Scope for published articles, ordered by most recent
scope :published, ->{ where(status: "published").order(published_at: :desc) }
# Scope for draft articles
scope :drafts, ->{ where(status: "draft") }
end
struct Post
includes CQL::ActiveRecord::Model(Int64)
includes CQL::ActiveRecord::Scopes # Important: Include the Scopes module
db_context AcmeDB, :posts
property created_at : Time
property category : String
property comment_count : Int32 = 0
# Scope for posts created after a certain date
scope :created_after, ->(date : Time) { where("created_at > ?", date) }
# Scope for posts in a specific category
scope :in_category, ->(category_name : String) { where(category: category_name) }
# Scope for limiting results (calls 'limit' on the current query chain)
scope :limited, ->(count : Int32) { limit(count) }
# Scope combining other scopes/query methods
scope :recent_in_category, ->(category_name : String, count : Int32 = 5) {
in_category(category_name).created_after(1.week.ago).limited(count).order(comment_count: :desc)
}
end
Using these scopes:
Article.published
returns a ChainableQuery(Article)
.
Post.created_after(7.days.ago)
returns a ChainableQuery(Post)
.
The scope
macro generates a class method. For example, scope :published, ->{ where(status: "published") }
on Article
is roughly equivalent to:
# def self.published
# # Proc is called, self inside proc refers to the current query object or class
# scope_call_result = (->{ where(status: "published") }).call
#
# if scope_call_result.is_a?(CQL::Query)
# ChainableQuery(Article).new(scope_call_result)
# else
# scope_call_result # Assumed to be ChainableQuery(Article)
# end
# end
This keeps your model definitions clean and focused on the query logic.
Alternatively, you can define scopes by creating class methods directly. This approach might be preferred for very complex logic that is hard to express in a single proc or if you prefer the explicitness of a full method definition.
When defining scopes as class methods, they should generally return a ChainableQuery(YourModel)
instance to maintain chainability. The Queryable
module (included via CQL::ActiveRecord::Model
) provides methods like where
, order
, etc., that return ChainableQuery(YourModel)
instances and can be used here.
Example: Basic Scopes as Class Methods
struct Article
includes CQL::ActiveRecord::Model(Int64)
# CQL::ActiveRecord::Scopes module is not strictly needed if only using class methods for scopes
db_context AcmeDB, :articles
property id : Int64?
property title : String
property status : String
property published_at : Time?
def self.published
where(status: "published").order(published_at: :desc)
end
def self.drafts
where(status: "draft")
end
end
Example: Scopes with Arguments as Class Methods
struct Post
includes CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :posts
property created_at : Time
property category : String
def self.created_after(date : Time)
where("created_at > ?", date)
end
def self.in_category(category_name : String)
where(category: category_name)
end
# Note: If you name a class method scope 'limit', it might hide the
# underlying Queryable#limit if not handled carefully. Using a different name like 'take'
# or being explicit (e.g. self.class.query.limit(c)) might be clearer in some cases.
def self.take(count : Int32)
limit(count) # Calls Queryable.limit available to the class
end
end
CQL::Query
(Less Common for Chaining)While ChainableQuery(YourModel)
is preferred for easy chaining (and is what the scope
macro ensures), a class method scope can return a raw CQL::Query
object. This is less common if you want to chain with other Active Record-style methods but might be useful for specific scenarios where you intend to pass the query object to a system expecting a base CQL::Query
.
struct Product
includes CQL::ActiveRecord::Model(Int64)
db_context AcmeDB, :products
property name : String?
property stock_count : Int32 = 0
# Scope returning a raw CQL::Query
def self.low_stock(threshold : Int32)
query.where("stock_count < ?", threshold) # Model.query returns a base CQL::Query
end
end
When a scope returns CQL::Query
, terminate it with methods like .all(Product)
or .first(Product)
.
Once defined (via macro or class method), scopes are called like any other class method:
# Using macro-defined scopes on Article model
published_articles = Article.published.all
puts "#{published_articles.size} published articles found."
# Using macro-defined scopes with arguments on Post model
recent_crystal_posts = Post.in_category("Crystal Lang").created_after(7.days.ago).limited(5).all
puts "#{recent_crystal_posts.size} recent Crystal Lang posts found."
# Using a class-method scope returning CQL::Query on Product model
low_stock_items = Product.low_stock(10).all(Product)
puts "#{low_stock_items.size} products with low stock."
Scopes returning ChainableQuery(YourModel)
are designed for chaining with each other and standard query methods (.where
, .order
, etc.):
# Chaining scopes on Article model
highly_viewed_published_articles = Article.published
.where("view_count > ?", 1000)
.order(view_count: :desc)
.limited(10) # Assuming a 'limited' scope or direct .limit call
.all
# Chaining scopes on Post model
featured_tech_posts = Post.in_category("Technology")
.created_after(1.month.ago)
.limited(3)
.order(comment_count: :desc)
.all
The ChainableQuery
class often uses forward_missing_to Target
(where Target
is your model class). This allows class methods on your model (including those generated by the scope
macro or defined manually) to be called on a ChainableQuery
instance, enabling natural chaining like Article.where(...).published
.
Scopes are a powerful feature for organizing your database query logic, making your application easier to read, write, and maintain. They promote the principle of keeping data logic within the model layer.