Only this pageAll pages
Powered by GitBook
1 of 41

Cql

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

CRUD Operations

Now that we have our Movie database schema ready we can proceed to create, read, update and delete records.

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Transaction Management

Loading...

Loading...

Loading...

Installation

To get started with CQL in your Crystal project, follow these steps:

Step 1: Add Dependency

First, add CQL to your project by including it in the shard.yml file:

dependencies:
  cql:
    github: azutoolkit/cql
    version: "~> 0.1.0"

Step 2: Install Shards

Run the following command to install the dependencies:

shards install

Step 3: Configure Database

Set up your database connection by specifying the adapter and connection URL. This is done by configuring the database in your code, as follows:

"postgres://user:password@localhost:5432/database_name"

In this example, we’re using PostgreSQL. You can change the URL according to your database (MySQL, SQLite, etc.).

Step 4: Create the Schema

Now you can define your schema and run migrations (explained in later sections).

README

CQL is a powerful library designed to simplify and enhance the management and execution of SQL queries in the Crystal programming language. It provides utilities for building, validating, and executing SQL statements, ensuring better performance and code maintainability.

Features

  • Query Builder: Programmatically create complex SQL queries.

  • Insert, Update, Delete Operations: Perform CRUD operations with ease.

  • Repository Pattern: Manage your data more effectively using CQL::Repository(T).

  • Active Record Pattern: Work with your data models using CQL::Record(T).

Installation

Add this to your application's shard.yml:

dependencies:
  cql:
    github: azutoolkit/cql

Then, run the following command to install the dependencies:

shards install

Getting Started

1. Define a Schema

Define the schema for your database tables:

AcmeDB2 = CQL::Schema.build(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]) do
  table :movies do
    primary :id, Int64, auto_increment: true
    text :title
  end

  table :screenplays do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :content
  end

  table :actors do
    primary :id, Int64, auto_increment: true
    text :name
  end

  table :directors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :name
  end

  table :movies_actors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    bigint :actor_id
  end
end

2. Executing Queries

With the schema in place, you can start executing queries:

q = AcmeDB.query
user = q.from(:users).where(id: 1).first!(as: User)
puts user.name

3. Inserting Data

Insert new records into the database:

q = CQL::Query.new(schema)
q.insert
  .into(:users)
  .values(name: "Jane Doe", email: "jane@example.com")
  .last_insert_id

4. Updating Data

Update existing records:

u = AcmeDB.update
u.table(:users)
  .set(name: "Jane Smith")
  .where(id: 1)
  .commit

5. Deleting Data

Delete records from the database:

d = AcmeDB.delete
d.from(:users).where(id: 1).commit

6. Using the Repository Pattern

Utilize the repository pattern for organized data management:

user_repository = CQL::Repository(User, Int64).new(schema, :users)

# Create a new user
user_repository.create(name: "Jane Doe", email: "jane@example.com")

# Fetch all users
users = user_repository.all
users.each { |user| puts user.name }

# Find a user by ID
user = user_repository.find!(1)
puts user.name

# Update a user by ID
user_repository.update(1, name: "Jane Smith")

7. Active Record Pattern

Work with your data using the Active Record pattern:

struct Actor < CQL::Record(Int64)

  db_context AcmeDB2, :actors

  getter id : Int64?
  getter name : String

  def initialize(@name : String)
  end
end

struct Movie < CQL::Record(Int64)

  db_context AcmeDB2, :movies

  has_one :screenplay, Screenplay
  many_to_many :actors, Actor, join_through: :movies_actors
  has_many :directors, Director, foreign_key: :movie_id

  getter id : Int64?
  getter title : String

  def initialize(@title : String)
  end
end

struct Director < CQL::Record(Int64)

  db_context AcmeDB2, :directors

  getter id : Int64?
  getter name : String
  belongs_to :movie, foreign_key: :movie_id

  def initialize(@name : String)
  end
end

struct Screenplay < CQL::Record(Int64)

  db_context AcmeDB2, :screenplays

  belongs_to :movie, foreign_key: :movie_id

  getter id : Int64?
  getter content : String

  def initialize(@movie_id : Int64, @content : String)
  end
end

struct MoviesActors < CQL::Record(Int64)

  db_context AcmeDB2, :movies_actors

  getter id : Int64?
  getter movie_id : Int64
  getter actor_id : Int64

  has_many :actors, Actor, :actor_id

  def initialize(@movie_id : Int64, @actor_id : Int64)
  end
end

Initializing Schema

Once you've defined your database schema using CQL's Schema.define, the next step is to initialize the database by creating the necessary tables and structures. In this guide, we will walk through how to define your schema and use Schema.init to initialize the database.


Real-World Example: Defining the Database Schema

Here's an example where we define a schema for a movie database using CQL. The schema includes tables for movies, screenplays, actors, directors, and a join table movies_actors to link movies and actors.

AcmeDB2 = CQL::Schema.define(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do
  table :movies do
    primary :id, Int64, auto_increment: true
    text :title
  end

  table :screenplays do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :content
  end

  table :actors do
    primary :id, Int64, auto_increment: true
    text :name
  end

  table :directors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :name
  end

  table :movies_actors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    bigint :actor_id
  end
end

Explanation

In the example above, we define:

  • movies: Stores information about movies, including an auto-incrementing id and a title.

  • screenplays: Stores the screenplay contents for movies, linking to the movies table with movie_id.

  • actors: Stores actor names, with an auto-incrementing id.

  • directors: Stores director names, associated with a movie via movie_id.

  • movies_actors: A join table to link movies and actors, establishing a many-to-many relationship between movies and actors.


Initializing the Database

Once the schema has been defined, the next step is to initialize the database. This is done by calling the Schema.init method (or in our case, AcmeDB2.init), which creates the tables based on the schema.

Steps to Initialize

  1. Set up the environment: Ensure that the DATABASE_URL environment variable is correctly set to point to your database connection string (for PostgreSQL in this case).

  2. Call Schema.init: After defining the schema, you initialize the database by calling the init method on the schema object.

AcmeDB2.init

This command creates all the tables and applies the structure you defined in the schema to your actual database.


Full Example: Defining and Initializing the Database

# db_context the schema
AcmeDB2 = CQL::Schema.define(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do
  table :movies do
    primary :id, Int64, auto_increment: true
    text :title
  end

  table :screenplays do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :content
  end

  table :actors do
    primary :id, Int64, auto_increment: true
    text :name
  end

  table :directors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :name
  end

  table :movies_actors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    bigint :actor_id
  end
end

# Initialize the database
AcmeDB2.init

Generated Database Tables

What Happens During Initialization?

When AcmeDB2.init is called, the following happens:

  • The database connection is established using the URI provided in the schema (e.g., the PostgreSQL database connection).

  • CQL creates the tables (movies, screenplays, actors, directors, and movies_actors) in the database if they don't already exist.

  • Primary keys, relationships, and any constraints are applied as defined in the schema.


Verifying the Initialization

After calling AcmeDB2.init, you can verify the tables are created in your database by using a PostgreSQL client or a GUI tool such as pgAdmin. You should see the tables with their respective columns and relationships as defined in the schema.

-- Example SQL to verify the tables were created
SELECT table_name FROM information_schema.tables WHERE table_schema = 'public';

-- To check the structure of a table, use:
\d movies  -- This shows the structure of the 'movies' table

Summary

Initializing the database after schema creation is a simple process with CQL. After defining your tables and relationships using CQL::Schema.define, you can call the init method to apply your schema to the actual database.

This method ensures that your database is correctly structured and ready to use, allowing you to focus on developing the application logic without worrying about manual database setup.

AcmeDB2.init  # Initializes the schema and sets up the database

Creating Records

The CQL::Insert class in the CQL (Crystal Query Language) module is a powerful tool designed to simplify the process of inserting records into a database. As a software developer, you'll find this class essential for building INSERT queries in a clean, readable, and chainable way.

In this guide, we’ll walk through the core functionality, explain each method in detail, and provide real-world examples to help you understand how to integrate CQL::Insert into your applications.


Key Features

  1. Insert records into any table with ease.

  2. Insert multiple records in a single query.

  3. Insert records using data fetched from another query.

  4. Get the last inserted ID after an insert.

  5. Chainable, intuitive syntax for building complex queries.


Real-World Example: Inserting User Data

Let’s start with a simple example of inserting a new user into the users table.

insert
  .into(:users)
  .values(name: "Alice", email: "alice@example.com", age: 29)
  .commit

This example demonstrates how you can insert a new user with their name, email, and age using the values method, followed by commit to execute the insert.


Core Methods

Let's dive into the individual methods and how you can use them.

1. into(table : Symbol)

Purpose: Specifies the table into which the data will be inserted. This is your starting point for any insert operation.

  • Parameter: table (Symbol) — The name of the table to insert into.

  • Returns: Insert object (enabling chaining).

Example:

insert.into(:users)

In the above code, we're targeting the users table. This is where subsequent data will be inserted.


2. values(**fields)

Purpose: Specifies the data (fields and values) to insert. The values method can accept either a hash or keyword arguments.

  • Parameter: fields (Hash or keyword arguments) — A mapping of column names to their values.

  • Returns: Insert object (for chaining).

Real-World Example 1: Adding a Single User

insert
  .into(:users)
  .values(name: "Bob", email: "bob@example.com", age: 35)
  .commit

Here, we’re adding a new user named Bob, specifying their name, email, and age.

Real-World Example 2: Adding Multiple Users in One Query

insert
  .into(:users)
  .values([
    {name: "John", email: "john@example.com", age: 30},
    {name: "Jane", email: "jane@example.com", age: 25}
  ]).commit

This example demonstrates how you can insert multiple users in a single query. It’s efficient and reduces database round trips.


3. last_insert_id(as type : PrimaryKeyType = Int64)

Purpose: Retrieves the ID of the last inserted row. This is incredibly useful when you need to work with the inserted record immediately after an insert, especially in cases where the primary key is automatically generated.

  • Parameter: type (default: Int64) — The data type of the returned ID.

  • Returns: The last inserted ID as Int64 or the specified type.

Example:

last_id = insert
  .into(:users)
  .values(name: "Charlie", email: "charlie@example.com", age: 22)
  .last_insert_id
puts last_id  # Outputs the last inserted ID

4. query(query : Query)

Purpose: Instead of manually specifying values, you can use data fetched from another query to populate the insert. This is useful in situations like copying data from one table to another.

  • Parameter: query — A query object that fetches the data to insert.

  • Returns: Insert object (for chaining).

Real-World Example: Copying Data from One Table to Another

Imagine you want to copy user data from an archive table (archived_users) to the main users table.

insert
  .into(:users)
  .query(select.from(:archived_users).where(active: true))
  .commit

In this example, we’re selecting all active users from archived_users and inserting them into the users table in one go.


5. back(*columns : Symbol)

Purpose: After an insert operation, you may want to return certain columns, like an ID or timestamp. The back method allows you to specify which columns should be returned.

  • Parameter: columns — One or more symbols representing the columns to return.

  • Returns: Insert object (for chaining).

Example:

insert
  .into(:users)
  .values(name: "David", email: "david@example.com", age: 28)
  .back(:id)
  .commit

In this case, after inserting the new user, we’re returning the id of the newly inserted row.


6. commit

Purpose: Executes the built INSERT query and commits the transaction to the database. This is the final step in your insert operation.

  • Returns: The result of the insert, typically the number of affected rows or the last inserted ID.

Example:

insert
  .into(:users)
  .values(name: "Eva", email: "eva@example.com", age: 31)
  .commit

This will execute the INSERT statement and commit it to the database, saving the new user’s data.


Advanced Example: Using Insert with Chained Operations

Let’s consider a more advanced example where you insert a new user, return their ID, and use it in subsequent operations.

user_id = insert
  .into(:users)
  .values(name: "Frank", email: "frank@example.com", age: 27)
  .last_insert_id

# Now use `user_id` in another query, e.g., to insert into a related table
insert
  .into(:user_profiles)
  .values(user_id: user_id, bio: "Software developer from NYC", twitter: "@frankdev")
  .commit

In this example, after inserting the new user, we immediately get the user_id and use it to insert data into the user_profiles table.


Handling Errors

In case an insert operation fails, the commit method automatically logs the error and raises an exception. This allows you to catch and handle the error as needed in your application.

Example:

begin
  insert
    .into(:users)
    .values(name: nil)  # This will likely fail due to a NOT NULL constraint
    .commit
rescue ex
  puts "Insert failed: #{ex.message}"
end

Conclusion

The CQL::Insert class provides a flexible, chainable interface for inserting data into a database, whether you're inserting a single record, multiple records, or even records based on the results of a query. Its intuitive API makes it easy to use in real-world applications, and with error handling built-in, it helps ensure robust database operations.

With its simple syntax and powerful features, CQL::Insert streamlines your database interactions, allowing you to focus on building great features in your Crystal applications.

Introduction

Purpose of CQL

CQL (Crystal Query Language) is a powerful tool designed for developers working with the Crystal programming language. It provides a streamlined interface for interacting with SQL databases, combining the flexibility of raw SQL with the safety and clarity of Crystal’s type system. CQL simplifies complex database operations, making it easier to perform CRUD (Create, Read, Update, Delete) operations, manage database schemas, and execute advanced queries.

Key Features

  • Database-agnostic: CQL supports multiple databases, ensuring flexibility in project development.

  • Active Record Pattern: Integrates seamlessly with Crystal structs, allowing developers to work with database records as native Crystal objects.

  • Query Builder: Provides an intuitive API for constructing complex SQL queries, including joins, subqueries, and transactions.

  • Associations: CQL supports defining relationships between tables (has_many, belongs_to, many_to_many), making it easy to navigate related data.

  • Migrations: Facilitates schema evolution by allowing developers to create and manage database migrations.

Supported Databases

CQL is designed to work with a range of SQL databases, including:

  • PostgreSQL: Full support with advanced features like JSONB, arrays, and full-text search.

  • MySQL: Support for common MySQL operations, though more advanced features may be limited.

  • SQLite: Lightweight database support for development and testing environments.

Use Cases

CQL is ideal for developers looking to integrate Crystal with SQL databases, providing a robust toolset for building data-driven applications. Whether you are developing a small-scale application or a large enterprise system, CQL offers the performance and scalability needed to manage your database operations efficiently.

Core Concepts

CQL’s core concepts revolve around providing developers with tools to efficiently interact with databases through well-structured APIs. These concepts are fundamental to building and manipulating data models, performing queries, and managing transactions. Here's a high-level overview:

  • Schemas: Define the structure of the database, including tables, columns, and data types.

  • CRUD Operations: Simplify creating, reading, updating, and deleting records in the database.

  • Query Builder: Enables constructing SQL queries using Crystal code with features like joins and subqueries.

  • Transactions: Ensure safe and atomic execution of multiple database operations.

Reading Records

The CQL::Query class is designed to simplify the creation and execution of SQL queries. By using a structured, chainable API, you can build complex SQL queries while maintaining clean and readable code.

This guide walks you through how to create, modify, and execute queries using real-world examples. We'll also explore various methods for selecting, filtering, joining, and ordering data, with a focus on practical use cases.


Key Features

  1. Select columns and filter records using simple, chainable methods.

  2. Join tables for complex queries involving multiple relationships.

  3. Aggregate data using functions like COUNT, SUM, and AVG.

  4. Order and limit your result sets for more precise control.

  5. Fetch results as objects or raw data for immediate use in your application.


Real-World Example: Fetching User Data

Let's begin by selecting user data from a users table:

query = CQL::Query.new(schema)
query.select(:name, :age).from(:users).where(name: "Alice").all(User)

This query will return all users with the name "Alice", casting the result to the User type.


Core Methods

Below is a breakdown of the key methods in the CQL::Query class and how you can use them in your applications.

1. select(*columns : Symbol)

Purpose: Specifies the columns to select in the query.

  • Parameters: columns — One or more symbols representing the columns you want to select.

  • Returns: Query object for chaining.

Real-World Example: Selecting Columns

query = CQL::Query.new(schema)
query.select(:name, :email).from(:users)

This query selects the name and email columns from the users table.


2. from(*tables : Symbol)

Purpose: Specifies the tables to query from.

  • Parameters: tables — One or more symbols representing the tables to query from.

  • Returns: Query object for chaining.

Real-World Example: Querying a Table

query.from(:users)

This query selects from the users table.


3. where(hash : Hash(Symbol, DB::Any))

Purpose: Adds filtering conditions to the query.

  • Parameters: hash — A key-value hash representing the column and its corresponding value for the WHERE clause.

  • Returns: Query object for chaining.

Real-World Example: Filtering Data

query.from(:users).where(name: "Alice", age: 30)

This will generate a query with the WHERE clause: WHERE name = 'Alice' AND age = 30.


4. all(as : Type)

Purpose: Executes the query and returns all matching results, casting them to the specified type.

  • Parameters: as — The type to cast the results to.

  • Returns: An array of the specified type.

Real-World Example: Fetching All Results

users = query.select(:name, :email).from(:users).all(User)

This will return all users as an array of User objects.


5. first(as : Type)

Purpose: Executes the query and returns the first matching result, casting it to the specified type.

  • Parameters: as — The type to cast the result to.

  • Returns: The first matching result of the query.

Real-World Example: Fetching the First Result

user = query.select(:name, :email).from(:users).where(name: "Alice").first(User)

This returns the first user with the name "Alice".


6. count(column : Symbol = :*)

Purpose: Adds a COUNT aggregate function to the query, counting the specified column.

  • Parameters: column — The column to count (default is *, meaning all rows).

  • Returns: Query object for chaining.

Real-World Example: Counting Rows

count = query.from(:users).count(:id).get(Int64)

This will count the number of rows in the users table and return the result as an Int64.


7. join(table : Symbol, on : Hash)

Purpose: Adds a JOIN clause to the query, specifying the table and the condition for joining.

  • Parameters:

    • table: The table to join.

    • on: A hash representing the join condition, mapping columns from one table to another.

  • Returns: Query object for chaining.

Real-World Example: Joining Tables

query
  .from(:users)
  .join(:orders, on: {users.id => orders.user_id})

This query joins the users table with the orders table on the condition that users.id equals orders.user_id.


8. order(*columns : Symbol)

Purpose: Specifies the columns by which to order the results.

  • Parameters: columns — The columns to order by.

  • Returns: Query object for chaining.

Real-World Example: Ordering Results

query.from(:users).order(:name, :age)

This orders the query results by name first and then by age.


9. limit(value : Int32)

Purpose: Limits the number of rows returned by the query.

  • Parameters: value — The number of rows to return.

  • Returns: Query object for chaining.

Real-World Example: Limiting Results

query.from(:users).limit(10)

This limits the query to return only the first 10 rows.


10. get(as : Type)

Purpose: Executes the query and returns a scalar value, such as the result of an aggregate function (e.g., COUNT, SUM).

  • Parameters: as — The type to cast the result to.

  • Returns: The scalar result of the query.

Real-World Example: Getting a Scalar Value

total_users = query.from(:users).count(:id).get(Int64)

This returns the total number of users as an Int64.


11. each(as : Type, &block)

Purpose: Iterates over each result, yielding each row to the provided block.

  • Parameters:

    • as: The type to cast each row to.

    • &block: The block to execute for each row.

  • Returns: Nothing (used for iteration).

Real-World Example: Iterating Over Results

query.from(:users).each(User) do |user|
  puts user.name
end

This will print the name of each user in the users table.


12. distinct

Purpose: Sets the DISTINCT flag to return only unique rows.

  • Returns: Query object for chaining.

Real-World Example: Fetching Distinct Results

query.from(:users).distinct

This will generate a query that returns only distinct rows from the users table.


Putting It All Together

Let's create a real-world example that combines several methods. Suppose you want to fetch the first 5 users who have placed an order, ordered by their name, and return the result as User objects:

query = CQL::Query.new(schema)

users = query
  .select(:name, :email)
  .from(:users)
  .join(:orders, on: {users.id => orders.user_id})
  .where(active: true)
  .order(:name)
  .limit(5)
  .all(User)

users.each do |user|
  puts "Name: #{user.name}, Email: #{user.email}"
end

In this query:

  • We select the name and email columns from users.

  • We join the orders table to ensure the user has placed an order.

  • We filter for active users (where(active: true)).

  • We order the results by name.

  • We limit the results to 5 users.


Conclusion

The CQL::Query class offers a flexible and intuitive API for building and executing SQL queries in your Crystal applications. With methods for selecting, joining, filtering, and aggregating data, you can handle even the most complex queries with ease.

Whether you're building basic queries or handling complex database operations, the CQL::Query class provides the tools you need to write clean, efficient, and maintainable code.

Altering Schema

The CQL::AlterTable class in the CQL framework provides a structured way to make schema changes to your database tables. It allows you to perform actions like adding, dropping, and renaming columns, as well as adding foreign keys, renaming tables, and creating indexes. This guide walks you through the most common use cases with real-world examples that software developers can apply in their projects.


Why Use CQL::AlterTable?

When your application evolves, you often need to modify your database structure. CQL::AlterTable lets you:

  • Add new columns to accommodate growing data needs.

  • Remove or rename columns as your data model refines.

  • Enforce foreign key relationships and maintain referential integrity.

  • Create or remove indexes for performance tuning.


Real-World Example: Modifying the Users Table

Let’s start with a basic example where we modify the users table by adding a new column email, removing the age column, and renaming the email column to user_email. Given the AcmeDB2 schema with a Users table:

AcmeDB2 = CQL::Schema.build(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do
  table :users do
    primary
    text name, size: 150
  end
end
alter = AlterTable.new(users_table, schema)

AcmeDB.alter :users do
  add_column(:email, "string", null: false, unique: true)
  drop_column(:age)
  rename_column(:email, :user_email)
end

This example:

  • Adds a new email column that cannot be NULL and must be unique.

  • Drops the age column, removing it from the table.

  • Renames the email column to user_email.


Core Methods in CQL::AlterTable

1. add_column(name : Symbol, type : Any, **options)

Purpose: Adds a new column to the table with flexible options for setting the column's properties, such as type, default value, uniqueness, etc.

  • Parameters:

    • name: The name of the column to add.

    • type: The data type of the column (e.g., String, Int32).

    • Additional options:

      • null: Whether the column can be NULL (default: true).

      • default: The default value for the column.

      • unique: Whether the column should have a unique constraint (default: false).

      • size: Optionally specify the size of the column (for strings or numbers).

      • index: Whether the column should be indexed (default: false).

Real-World Example: Adding an Email Column

alter.add_column(:email, "string", null: false, unique: true)

This adds an email column to the table, ensures it is NOT NULL, and enforces a uniqueness constraint.


2. drop_column(name : Symbol)

Purpose: Removes an existing column from the table.

  • Parameters:

    • name: The name of the column to drop.

Real-World Example: Dropping a Column

alter.drop_column(:age)

This removes the age column from the table.


3. rename_column(old_name : Symbol, new_name : Symbol)

Purpose: Renames an existing column in the table.

  • Parameters:

    • old_name: The current name of the column.

    • new_name: The new name for the column.

Real-World Example: Renaming a Column

alter.rename_column(:email, :user_email)

This renames the email column to user_email.


4. change_column(name : Symbol, type : Any)

Purpose: Changes the data type of an existing column.

  • Parameters:

    • name: The name of the column.

    • type: The new data type for the column.

Real-World Example: Changing a Column’s Type

alter.change_column(:age, "string")

This changes the age column’s type from whatever it was (likely an Int32) to a string.


5. rename_table(new_name : Symbol)

Purpose: Renames the entire table.

  • Parameters:

    • new_name: The new name for the table.

Real-World Example: Renaming a Table

alter.rename_table(:customers)

This renames the table from users to customers.


6. foreign_key(name : Symbol, columns : Array(Symbol), table : Symbol, references : Array(Symbol), **options)

Purpose: Adds a foreign key constraint to the table.

  • Parameters:

    • name: The name of the foreign key.

    • columns: The columns in the current table to use as the foreign key.

    • table: The referenced table.

    • references: The columns in the referenced table.

    • Options:

      • on_delete: Action to take on delete (default: NO ACTION).

      • on_update: Action to take on update (default: NO ACTION).

Real-World Example: Adding a Foreign Key

alter.foreign_key(:fk_movie_id, [:movie_id], :movies, [:id], on_delete: "CASCADE")

This adds a foreign key fk_movie_id on the movie_id column, linking it to the id column in the movies table. On delete, it cascades the delete.


7. drop_foreign_key(name : Symbol)

Purpose: Removes a foreign key constraint from the table.

  • Parameters:

    • name: The name of the foreign key to remove.

Real-World Example: Dropping a Foreign Key

alter.drop_foreign_key(:fk_movie_id)

This drops the foreign key constraint fk_movie_id from the table.


8. create_index(name : Symbol, columns : Array(Symbol), unique : Bool = false)

Purpose: Adds an index to the specified columns.

  • Parameters:

    • name: The name of the index.

    • columns: The columns to index.

    • unique: Whether the index should enforce uniqueness (default: false).

Real-World Example: Creating a Unique Index

alter.create_index(:index_users_on_email, [:email], unique: true)

This creates a unique index on the email column, ensuring that email addresses are unique across the table.


9. drop_index(name : Symbol)

Purpose: Removes an index from the table.

  • Parameters:

    • name: The name of the index to remove.

Real-World Example: Dropping an Index

alter.drop_index(:index_users_on_email)

This drops the index index_users_on_email from the table.


10. to_sql(visitor : Expression::Visitor)

Purpose: Generates the SQL string corresponding to the current set of table alterations.

  • Parameters:

    • visitor: The SQL generator that converts the actions to SQL.

Real-World Example: Generating SQL for Table Alterations

sql = alter.to_sql(visitor)
puts sql

This will generate the SQL statements that correspond to the actions taken (e.g., adding columns, dropping columns).


Putting It All Together

Here’s an advanced example where we modify the users table by adding and removing columns, renaming the table, and creating an index:

alter = AlterTable.new(users_table, schema)

sql = AcmeDB2.alter :users do
  add_column(:email, "string", null: false, unique: true)
  drop_column(:age)
  rename_table(:customers)
  create_index(:index_customers_on_email, [:email], unique: true)
end.to_sql

puts sql

This code:

  1. Adds a email column with NOT NULL and UNIQUE constraints.

  2. Drops the age column.

  3. Renames the users table to customers.

  4. Creates a unique index on the email column.

  5. Generates the SQL statements that implement these actions.


Conclusion

The CQL::AlterTable class provides a simple and flexible interface for modifying your database schema. With its chainable methods, you can easily add, drop, or modify columns, rename tables, and manage foreign keys and indexes. This makes managing your database schema effortless, allowing you to focus on building robust, scalable applications.

Migrations

Database migrations are essential for managing changes to your schema over time in a controlled manner. In CQL, migrations are handled through the Migration and Migrator classes. This guide will help you understand how to create, apply, rollback, and manage migrations using CQL::Migrator in your projects.

Why Use Migrations?

Migrations allow you to:

  • Apply changes to your database schema over time.

  • Roll back changes in case of errors or updates.

  • Track applied and pending changes, ensuring consistency across environments.


Real-World Example: Creating and Applying Migrations

Let’s start with a simple example. Suppose we need to add a users table to our database with two columns: name and age.

Explanation

  • The up method: defines the changes to apply when the migration is run (e.g., adding new columns).

  • The down method: defines how to revert the changes (e.g., dropping columns).

  • Versioning: Each migration is assigned a version number, which ensures migrations are run in the correct order.


Initializing the Schema and Migrator

Before applying migrations, you need to set up the schema and create an instance of the Migrator.

The migrator, upon initialization, automatically creates a schema_migrations table to track which migrations have been applied.


Applying Migrations

To apply all pending migrations, simply call the up method on the migrator object:

This will apply all pending migrations in order of their version numbers.

Applying Migrations Up to a Specific Version

You can also apply migrations up to a specific version:

This will apply all migrations up to version 1_i64.


Rolling Back Migrations

To roll back the last migration, use the down method:

You can also roll back to a specific migration version:

This rolls back all migrations down to version 1_i64.


Redoing Migrations

If you want to rollback and then re-apply the last migration, use the redo method:

This first rolls back the last migration and then re-applies it.


Listing Migrations

You can list applied, pending, and rolled-back migrations with the following commands:

  • List Applied Migrations:

  • List Pending Migrations:

  • List Rolled Back Migrations:

These commands provide a clear view of the current state of your migrations, making it easy to track progress and issues.


Managing Migrations

Checking the Last Applied Migration

You can retrieve information about the last applied migration using:

This gives you details about the last migration that was successfully applied.


Advanced Example: Managing Multiple Migrations

Here’s an example where we define multiple migrations and apply them sequentially:

  • Versioning ensures that migrations are applied in the correct order.

  • Each migration can be applied and rolled back independently, offering flexibility in managing your database schema.


Conclusion

The CQL::Migrator class makes it easy to manage database migrations in a structured and version-controlled manner. By following this guide, you can:

  • Create and apply migrations to modify your schema.

  • Roll back changes if needed.

  • Track applied and pending migrations to keep your database consistent across environments.

This approach is essential for teams working on large applications where database changes need to be applied safely and consistently over time.

class CreateUsersTable < CQL::Migration(1)
  def up
    schema.alter :users do
      add_column :name, String
      add_column :age, Int32
    end
  end

  def down
    schema.alter :users do
      drop_column :name
      drop_column :age
    end
  end
end
schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
  ...
end

migrator = CQL::Migrator.new(schema)
migrator.up
migrator.up_to(1_i64)
migrator.down
migrator.down_to(1_i64)
migrator.redo
migrator.print_applied_migrations
migrator.print_pending_migrations
migrator.print_rolled_back_migrations
last_migration = migrator.last
puts last_migration
class CreateMoviesTable < CQL::Migration(2)
  def up
    schema.alter :movies do
      add_column :title, String
      add_column :release_year, Int32
    end
  end

  def down
    schema.alter :movies do
      drop_column :title
      drop_column :release_year
    end
  end
end

class CreateActorsTable < CQL::Migration
  self.version = 3_i64

  def up
    schema.alter :actors do
      add_column :name, String
    end
  end

  def down
    schema.alter :actors do
      drop_column :name
    end
  end
end

# Apply the migrations
migrator.up

Deleting Records

The CQL::Delete class provides a structured and flexible way to build and execute SQL DELETE queries in your Crystal applications. This guide will help you understand how to create delete queries, apply conditions, and execute them to remove records from your database.


Key Features

  1. Delete records from any table in a straightforward manner.

  2. Filter records to delete using flexible WHERE conditions.

  3. Return columns after deletion if needed.

  4. Chainable syntax for clean and maintainable query building.


Real-World Example: Deleting a User Record

Let’s start with a simple example of deleting a user from the users table where the id is 1.

delete = CQL::Delete.new(schema)
  .from(:users)
  .where(id: 1)
  .commit

This query deletes the record in the users table where id = 1.


Core Methods

The following section provides a breakdown of the key methods available in the CQL::Delete class and how to use them effectively.

1. from(table : Symbol)

Purpose: Specifies the table from which records will be deleted.

  • Parameters: table — A symbol representing the table name.

  • Returns: Delete object (for chaining).

Real-World Example: Specifying the Table

delete.from(:users)

This sets the users table as the target for the delete operation.


2. where(**fields)

Purpose: Adds a WHERE clause to filter the records to be deleted.

  • Parameters: fields — A key-value hash where keys represent column names and values represent the conditions to match.

  • Returns: Delete object (for chaining).

Real-World Example: Filtering by Conditions

delete
  .from(:users)
  .where(id: 1)

This filters the query to only delete the user where id = 1.


3. where(&block)

Purpose: Adds a WHERE clause using a block for more complex filtering conditions.

  • Parameters: A block that defines the filtering logic using a filter builder.

  • Returns: Delete object (for chaining).

Real-World Example: Using a Block for Conditions

delete
  .from(:users)
  .where { |w| w.age < 30 }

This deletes all users where the age is less than 30.


4. commit

Purpose: Executes the delete query and commits the changes to the database.

  • Returns: A DB::Result object representing the result of the query execution.

Real-World Example: Committing the Delete

delete = CQL::Delete.new(schema)
  .from(:users)
  .where(id: 1)
  .commit

This deletes the user from the users table where id = 1 and commits the change.


5. using(table : Symbol)

Purpose: Adds a USING clause to the delete query, useful when deleting records based on conditions from another table.

  • Parameters: table — A symbol representing the name of the table to use in the USING clause.

  • Returns: Delete object (for chaining).

Real-World Example: Using Another Table for Deletion

delete
  .from(:users)
  .using(:posts)
  .where { |w| w.posts.user_id == w.users.id }

This example deletes users where they are linked to posts based on the condition posts.user_id = users.id.


6. back(*columns : Symbol)

Purpose: Specifies the columns to return after the delete operation.

  • Parameters: columns — An array of symbols representing the columns to return.

  • Returns: Delete object (for chaining).

Real-World Example: Returning Columns After Deletion

delete
  .from(:users)
  .where(id: 1)
  .back(:name, :email)
  .commit

This deletes the user with id = 1 and returns the name and email of the deleted record.


7. to_sql(gen = @schema.gen)

Purpose: Generates the SQL query and parameters required for the delete operation.

  • Parameters: gen — The generator used for SQL generation (default: schema generator).

  • Returns: A tuple containing the SQL query string and the parameters.

Real-World Example: Generating SQL for Deletion

delete = CQL::Delete.new(schema)
  .from(:users)
  .where(id: 1)

sql, params = delete.to_sql
puts sql     # "DELETE FROM users WHERE id = $1"
puts params  # [1]

This generates the raw SQL query and its associated parameters without executing it.


Putting It All Together

Let’s combine multiple methods to handle a more advanced use case. Suppose you want to delete a user from the users table where they have no associated posts, and you want to return the deleted user’s name and email:

delete = CQL::Delete.new(schema)

result = delete
  .from(:users)
  .using(:posts)
  .where { |w| w.posts.user_id == w.users.id && w.posts.id.nil? }
  .back(:name, :email)
  .commit

puts result  # This returns the name and email of the deleted user(s).

In this query:

  • We specify the users table as the target for deletion.

  • We use the posts table to filter users without any posts.

  • We return the name and email of the deleted user(s).


Conclusion

The CQL::Delete class provides an intuitive and powerful interface for deleting records in your Crystal applications. With chainable methods for setting conditions, joining tables, and selecting return columns, you can easily construct and execute delete queries with precision and clarity.

Whether you need to delete specific records or perform complex, condition-based deletions, the CQL::Delete class ensures that your queries are efficient and maintainable.

Active Record

Active Record is a design pattern used in Object-Relational Mapping (ORM) that simplifies database access by linking database tables directly to classes in your application. Each class represents a table, and each instance of the class corresponds to a row in that table. Active Record makes it easy to perform CRUD (Create, Read, Update, Delete) operations on the database.

In the context of CQL (Crystal Query Language) and Crystal, the Active Record pattern can be implemented by leveraging the object-oriented nature of Crystal and the querying capabilities provided by CQL. Here's how you can think about the Active Record pattern using CQL:

Key Concepts of Active Record

  1. Table-Class Mapping: Each class corresponds to a table in the database.

  2. Row-Object Mapping: Each object is an instance of a class and corresponds to a row in the database table.

  3. Database Operations as Methods: Methods on the object handle database interactions (e.g., .save, .delete, .find).

  4. Associations: Relationships between tables (e.g., belongs_to, has_many) are handled within the class structure.

  5. Validation: Logic to ensure data integrity is embedded within the class.

Implementing Active Record in CQL

1. Define a Model

In Active Record, a model is a class that represents a database table. The class will contain attributes (columns), methods for interacting with the data, and associations.

Here’s an example of a User model:

struct User
  include CQL::ActiveRecord::Model(Int32)
  # Schame name AcmeDB, table name :users
  db_context AcmeDB, :users

  property id : Int32?
  property name : String
  property email : String
  property created_at : Time
  property updated_at : Time
end

In this example:

  • struct User is used instead of class User

  • Include CQL::Record(User, Int32) specifies that User is the model and the primary key is of type Int32.

  • The model still contains the properties (id, name, email, created_at, updated_at), but now we delegate all Active Record-like operations (e.g., save, delete) to CQL::Record.

CQL makes no assumptions about table names and it must be explicitly provided. Schama name AcmeDB, Table name :users

define AcmeDB, :users

2. Performing CRUD Operations

In the Active Record pattern, CRUD operations (Create, Read, Update, Delete) are performed directly on the class and instance methods. Here’s how you can implement CRUD with CQL:

Create (INSERT)

To create a new record in the database, instantiate a new object of the model class and call .save to persist it:

user = User.new
user.name = "John Doe"
user.email = "john.doe@example.com"
user.created_at = Time.now
user.updated_at = Time.now
user.save # INSERTS the new user into the database

This will generate an INSERT INTO SQL statement and persist the user in the users table.

Read (SELECT)

To retrieve records from the database, you can use class-level methods like .find or .all. For example:

  • Fetch all users:

users = User.all # SELECT * FROM users
  • Find a user by id:

user = User.find(1) # SELECT * FROM users WHERE id = 1

Update

To update an existing record, modify the object and call .save again. This will generate an UPDATE SQL statement:

user = User.find(1)
user.name = "Jane Doe"
user.save # UPDATE users SET name = 'Jane Doe' WHERE id = 1

Delete

To delete a record, find the object and call .delete:

user = User.find(1)
user.delete # DELETE FROM users WHERE id = 1

3. Associations

Active Record also simplifies relationships between tables, such as has_many and belongs_to. In CQL, you can implement these relationships like this:

has_many (One-to-Many Relationship)

A User has many Posts. You can db_context the association like this:

struct User < CQL::Record(User, Int32)

  property id : Int32?
  property name : String
  property email : String
  property created_at : Time
  property updated_at : Time

  has_many :posts, Post
end

struct Post
  include CQL::ActiveRecord::Model(Int32)

  property id : Int32?
  property title : String
  property body : String
  property created_at : Time
  property updated_at : Time

  belongs_to :user, User
end

Now you can fetch the posts for a user:

user = User.find(1)
posts = user.posts # SELECT * FROM posts WHERE user_id = 1

belongs_to (Many-to-One Relationship)

The Post class has a belongs_to relationship with User. This means each post belongs to a user:

post = Post.find(1)
user = post.user # SELECT * FROM users WHERE id = post.user_id

Managing HasMany and ManyToMany Collections

Here is a summary of the collection methods provided in the CQL::Relations::Collection and CQL::Relations::ManyCollection classes for managing associations in a one-to-many and many-to-many relationship in CQL:

Collection Methods

  1. all:

    • Returns all associated records for the parent record.

    • Example: movie.actors.all

  2. reload:

    • Reloads the associated records from the database.

    • Example: movie.actors.reload

  3. ids:

    • Returns a list of primary keys for the associated records.

    • Example: movie.actors.ids

  4. <<:

    • Adds a new record to the association and persists it to the database.

    • Example: movie.actors << Actor.new(name: "Laurence Fishburne")

  5. empty?:

    • Checks if the association has any records.

    • Example: movie.actors.empty?

  6. **exists?(**attributes)**:

    • Checks if any associated records exist that match the given attributes.

    • Example: movie.actors.exists?(name: "Keanu Reeves")

  7. size:

    • Returns the number of associated records.

    • Example: movie.actors.size

  8. **find(**attributes)**:

    • Finds associated records that match the given attributes.

    • Example: movie.actors.find(name: "Keanu Reeves")

  9. **create(**attributes)**:

    • Creates a new record with the provided attributes and associates it with the parent.

    • Example: movie.actors.create(name: "Carrie-Anne Moss")

  10. create!(record):

    • Creates and persists a new record with the provided attributes, raising an error if it fails.

    • Example: movie.actors.create!(name: "Hugo Weaving")

  11. ids=(ids : Array(Pk)):

    • Associates the parent record with the records that match the provided primary keys.

    • Example: movie.actors.ids = [1, 2, 3]

  12. delete(record : Target):

    • Deletes the associated record from the parent record if it exists.

    • Example: movie.actors.delete(Actor.find(1))

  13. delete(id : Pk):

    • Deletes the associated record by primary key.

    • Example: movie.actors.delete(1)

  14. clear:

    • Removes all associated records for the parent record.

    • Example: movie.actors.clear

ManyCollection Additional Methods

  • In addition to the methods inherited from Collection, the ManyCollection class also manages associations through a join table in many-to-many relationships.

  1. create(record : Target):

    • Associates the parent record with the created record through a join table.

    • Example: movie.actors.create!(name: "Carrie-Anne Moss")

  2. delete(record : Target):

    • Deletes the association through the join table between the parent and associated record.

    • Example: movie.actors.delete(Actor.find(1))

  3. ids=(ids : Array(Pk)):

    • Associates the parent record with the records matching the primary keys through the join table.

    • Example: movie.actors.ids = [1, 2, 3]

These methods provide powerful ways to interact with and manage associations between records in a CQL-based application using both one-to-many and many-to-many relationships.

4. Validation

Active Record often includes validations to ensure that data meets certain criteria before saving. In CQL, you can add custom validation logic inside the class:

struct User
  include CQL::ActiveRecord::Model(Int32)

  property id : Int32?
  property name : String
  property email : String
  property created_at : Time
  property updated_at : Time

  def validate
    raise "Name can't be empty" if name.empty?
    raise "Email can't be empty" if email.empty?
  end

  def save
    validate
    super # Calls the original save method provided by CQL::Entity
  end
end

In this example, before saving a user, the validate method is called to ensure that the name and email are not empty.

Alternatively validations can be supported by other shards. For example Schema shard from the Azu Toolkit can be use to db_context validations

5. Migrations

Although not strictly part of Active Record, migrations are commonly used with it to modify database schemas over time. In CQL, migrations can be written using a migration system to create and alter tables. For example:

class CreateUsersTable < CQL::Migration
  self.version = 1_i64

  def up
    schema.alter :users do
      add_column :name, String
      add_column :age, Int32
    end
  end

  def down
    schema.alter :users do
      drop_column :name
      drop_column :age
    end
  end
end

This migration would create the users table with the specified columns.

How CQL Implements Active Record Principles

  • Model Representation: Each class (like User, Post) maps directly to a database table.

  • CRUD Operations: Operations like .save, .delete, .find, and .all are built into the CQL framework, allowing for seamless interaction with the database.

  • Associations: Relationships between models are defined using macros like has_many and belongs_to, which make querying associated records straightforward.

  • Encapsulation of Business Logic: Validation and other business rules can be embedded directly into model classes.

  • Database Migrations: Schema changes are managed through migrations, which help keep the database structure synchronized with the application's models.

Example Workflow with Active Record and CQL

  1. db_context your models:

    • Create User and Post classes that correspond to users and posts tables.

  2. Run Migrations:

    • Use migrations to create or modify the database schema.

  3. Perform CRUD operations:

    • Create, read, update, and delete records using model methods like .save and .delete.

  4. Manage relationships:

    • db_context associations like has_many and belongs_to to handle relationships between models.

  5. Enforce business rules:

    • Use validation methods to ensure data integrity.

By following the Active Record pattern in CQL, you can build a robust data access layer in your Crystal application with minimal effort while keeping your code clean and maintainable.

Repository

The Repository pattern is a design pattern that mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects. It centralizes data access logic, promoting a cleaner separation of concerns and making the application easier to test and maintain.

In CQL (Crystal Query Language), while the Active Record pattern is a prominent feature, CQL's design is flexible enough to support the Repository pattern. This allows developers to abstract the underlying data persistence mechanisms further and work with domain objects in a more decoupled manner.

Key Concepts of the Repository Pattern

  1. Abstraction of Data Access: The repository provides an abstraction layer over data storage. Consumers of the repository work with domain objects and are unaware of how those objects are persisted or retrieved.

  2. Collection-Like Interface: Repositories often expose methods that resemble collection operations, such as add, remove, find, all, etc.

  3. Centralized Query Logic: Queries related to a specific aggregate root or entity are encapsulated within its repository.

  4. Decoupling: It decouples the domain model from the data access concerns, improving testability and maintainability.

  5. Unit of Work (Optional but Common): Repositories are often used in conjunction with a Unit of Work pattern to manage transactions and track changes to objects.

Implementing the Repository Pattern with CQL

CQL provides a generic CQL::Repository(T, Pk) class that can be used as a base for creating specific repositories for your domain entities.

1. Define a Model (Entity)

First, you need a model (often a plain Crystal struct or class) that represents your domain entity. Unlike Active Record models, these entities typically don't include persistence logic themselves.

# Example: A simple User entity
struct User
  property id : Int32?
  property name : String
  property email : String

  def initialize(@name : String, @email : String, @id : Int32? = nil)
  end
end

2. Create a Specific Repository

You create a repository for your entity by inheriting from CQL::Repository(T, Pk), where T is your entity type and Pk is the type of its primary key.

require "cql"

# Define your schema (database connection)
MySchema = CQL::Schema.define(
  :my_app_db,
  adapter: CQL::Adapter::Postgres, # or any other supported adapter
  uri: ENV["DATABASE_URL"]? || "postgres://localhost:5432/my_app_dev"
)

# Define the User entity (as above)
struct User
  property id : Int32?
  property name : String
  property email : String

  def initialize(@name : String, @email : String, @id : Int32? = nil)
  end

  # Constructor to map from a Hash, often useful with database results
  def self.new(attrs : Hash(Symbol, DB::Any))
    new(
      name: attrs[:name].as(String),
      email: attrs[:email].as(String),
      id: attrs[:id]?.as?(Int32)
    )
  end
end

# UserRepository inheriting from CQL::Repository
class UserRepository < CQL::Repository(User, Int32)
  # The constructor takes the schema instance and the table name
  def initialize(schema : CQL::Schema = MySchema, table_name : Symbol = :users)
    super(schema, table_name)
  end

  # You can add custom query methods here
  def find_active_users
    query.where { active == true }.all(User)
  end

  def find_by_email_domain(domain : String)
    query.where { email.like("%@#{domain}") }.all(User)
  end
end

Explanation:

  • UserRepository < CQL::Repository(User, Int32): Defines a repository for User entities where the primary key is Int32.

  • initialize(schema : CQL::Schema = MySchema, table_name : Symbol = :users): The constructor expects a CQL::Schema instance and the symbol representing the database table name (e.g., :users).

  • super(schema, table_name): Calls the constructor of the base CQL::Repository class.

  • find_active_users, find_by_email_domain: Custom methods that encapsulate specific query logic using CQL's query builder (query.where(...)). The query method is provided by the base CQL::Repository.

3. Using the Repository

Once defined, you can use the repository to interact with your data:

# Initialize the repository
user_repo = UserRepository.new

# Create a new user
# The `create` method in the base Repository expects a Hash of attributes
new_user_id = user_repo.create({name: "Alice Wonderland", email: "alice@example.com"})
puts "Created user with ID: #{new_user_id}"

# Find a user by ID
alice = user_repo.find(new_user_id.as(Int32))
if alice
  puts "Found user: #{alice.name}"
else
  puts "User not found."
end

# Fetch all users
all_users = user_repo.all
puts "All users:"
all_users.each do |user|
  puts "- #{user.name} (#{user.email})"
end

# Update a user
if alice
  user_repo.update(alice.id.not_nil!, {email: "alice.wonderland@newdomain.com"})
  updated_alice = user_repo.find(alice.id.not_nil!)
  puts "Updated email: #{updated_alice.try &.email}"
end

# Use custom repository methods
active_users = user_repo.find_active_users # Assuming an 'active' column and logic
puts "Active users: #{active_users.size}"

# Delete a user
if alice
  user_repo.delete(alice.id.not_nil!)
  puts "Deleted user #{alice.name}"
end

Methods Provided by CQL::Repository(T, Pk)

The base CQL::Repository class provides several convenient methods for common data operations:

  • query: Returns a new CQL::Query instance scoped to the repository's table.

  • insert: Returns a new CQL::Insert instance scoped to the repository's table.

  • update: Returns a new CQL::Update instance scoped to the repository's table.

  • delete: Returns a new CQL::Delete instance scoped to the repository's table.

  • build(attrs : Hash(Symbol, DB::Any)) : T: Instantiates a new entity T with the given attributes (does not persist).

  • all : Array(T): Fetches all records.

  • find(id : Pk) : T?: Finds a record by its primary key, returns nil if not found.

  • find!(id : Pk) : T: Finds a record by its primary key, raises DB::NoResultsError if not found.

  • **find_by(**fields) : T?**: Finds the first record matching the given field-value pairs.

  • **find_all_by(**fields) : Array(T)**: Finds all records matching the given field-value pairs.

  • create(attrs : Hash(Symbol, DB::Any)) : Pk: Creates a new record with the given attributes and returns its primary key.

  • **create(**fields) : Pk**: Creates a new record with the given attributes (using named arguments) and returns its primary key.

  • update(id : Pk, attrs : Hash(Symbol, DB::Any)): Updates the record with the given id using the provided attributes.

  • update(id : Pk, **fields): Updates the record with the given id using the provided attributes (named arguments).

  • update_by(where_attrs : Hash(Symbol, DB::Any), update_attrs : Hash(Symbol, DB::Any)): Updates records matching where_attrs with update_attrs.

  • update_all(attrs : Hash(Symbol, DB::Any)): Updates all records in the table with the given attributes.

  • delete(id : Pk): Deletes the record with the given primary key.

  • **delete_by(**fields)**: Deletes records matching the given field-value pairs.

  • delete_all: Deletes all records in the table.

  • count : Int64: Returns the total number of records in the table.

  • **exists?(**fields) : Bool**: Checks if any records exist matching the given field-value pairs.

Advantages of Using the Repository Pattern with CQL

  • Improved Testability: You can easily mock repositories in your tests, isolating your domain logic from the database.

  • Centralized Data Access Logic: Keeps data access concerns separate from your domain entities and application services.

  • Flexibility: While CQL::Repository provides a good starting point, you can customize it or even write your own repository implementations if needed, without altering your domain models significantly.

  • Clearer Intent: Queries are named and reside within the repository, making their purpose more explicit.

When to Use the Repository Pattern

  • In larger applications where a clear separation between domain logic and data access is crucial.

  • When you need to support multiple data sources or switch data storage technologies with minimal impact on the domain layer.

  • When your querying logic becomes complex and you want to encapsulate it.

  • To improve the testability of your application by allowing easier mocking of data access.

While Active Record can be simpler for straightforward CRUD operations, the Repository pattern offers more structure and decoupling for complex applications, and CQL provides the tools to implement it effectively.

Schema Definition

Accelerating Database Iteration

Defining the schema first is a fundamental approach in CQL, helping developers quickly structure their database while keeping their application’s data model in sync with real-world entities. By defining your schema upfront, you can rapidly iterate over your database tables, making it easy to adjust data structures as your application evolves. This method ensures that your schema is the single source of truth, giving you a clear view of how your data is organized and how relationships between different tables are modeled.

Benefits of Defining the Schema First

  1. Faster Prototyping: With schemas defined at the outset, you can rapidly experiment with different table structures and relationships, making it easier to adjust your application’s data model without writing complex migrations from scratch.

  2. Clear Data Structure: When your schema is predefined, the application’s data structure becomes clearer, allowing developers to conceptualize how data is organized and interact with tables more easily.

  3. Consistency: Ensuring the schema matches the database at all times removes ambiguity when writing queries, handling relationships, or performing migrations.

  4. Automatic Data Validation: CQL schemas enforce data types and constraints, such as primary, auto_increment, and text, ensuring data integrity.

  5. Simplified Query Building: Since the schema is explicit, writing queries becomes easier as you can reference schema objects directly in queries, avoiding mistakes or typos in table or column names.

Difference from Other ORM Libraries

Unlike traditional ORM libraries (e.g., Active Record in Rails or Ecto in Elixir), which often allow defining database models alongside the code and handling schema evolution through migrations, CQL encourages defining the database schema as the first step.

This "schema-first" approach differs from the "code-first" or "migration-based" methodologies in that it avoids relying on automatic migrations or conventions to infer the structure of the database. CQL enforces an explicit and structured approach to schema creation, ensuring the database schema reflects the actual architecture of your application.

Example Schema Definition

Here’s a basic example of how to define a schema in CQL for a movie-related database:

Explanation of Schema Definition

  • Database name: :acme_db defines the schema name.

  • Adapter: CQL::Adapter::Postgres specifies the database adapter (in this case, PostgreSQL).

  • Connection URL: The uri: ENV["DATABASE_URL"] specifies the database connection using environment variables.

Each table is explicitly defined with its columns, such as:

  • :movies table has id as the primary key and title as a text column.

  • :screenplays, :actors, and :directors define relationships between movies and associated records.

This example shows how easy it is to define tables and manage relationships within the schema, leading to a more organized and coherent database structure that aligns with the application’s needs.

Multiple Schemas: Flexibility and Easy Switching

One significant advantage of CQL is the ability to define and manage multiple schemas within the same application. This is particularly useful in scenarios like multi-tenant applications, where each tenant or environment has a separate database schema. CQL makes switching between schemas seamless, enabling developers to organize different parts of the application independently while maintaining the same connection configuration.

This approach offers the following benefits:

  • Clear Separation of Data: Each schema can encapsulate its own set of tables and relationships, allowing better isolation and separation of concerns within the application. For example, you might have a main schema for core business data and a separate analytics schema for reporting.

  • Simple Switching: Switching between schemas is as simple as referring to the schema name, thanks to CQL’s structured definition of schemas. This allows dynamic switching at runtime, improving scalability in multi-tenant applications.

Example: Managing Multiple Schemas

In this example, you define multiple schemas, and the application can easily switch between MainDB and AnalyticsDB depending on which database needs to be queried.

Benefits of Multiple Schemas

  • Improved Organization: Separate business logic data from other concerns like reporting, testing, or archiving.

  • Scalability: Ideal for multi-tenant applications, allowing each tenant to have its schema without interference.

By using CQL’s schema system, you gain not only speed and clarity in your database structure but also flexibility in scaling and organizing your application.

CRUD Operations

CQL Active Record models provide a rich set of methods for performing CRUD (Create, Read, Update, Delete) operations on your database records. These methods are designed to be intuitive and align with common Active Record patterns.

This guide assumes you have a model defined, for example:


1. Creating Records

There are several ways to create new records and persist them to the database.

new + save / save!

The most fundamental way is to create a new instance of your model using new and then call save or save! to persist it.

  • save: Attempts to save the record. Runs validations. If successful, populates the id (if auto-generated) and returns true. If validations fail or another before_save callback halts the chain, it returns false and does not persist the record. The errors collection on the instance can be inspected.

  • save!: Similar to save, but raises an exception if the record cannot be saved.

    • Raises CQL::Errors::RecordInvalid if validations fail.

    • May raise other database-related exceptions or CQL::Errors::RecordNotSaved for other save failures.

create / create! (Class Methods)

These class methods provide a convenient way to instantiate and save a record in a single step.

  • Model.create(attributes): Creates a new instance with the given attributes (as a Hash or keyword arguments) and attempts to save it. Returns the model instance (which might be invalid and not persisted if save failed) or false if a before_create callback halted the process.

  • Model.create!(attributes): Similar to create, but calls save! internally. It will raise an exception (CQL::Errors::RecordInvalid or other) if the record cannot be created and persisted.

find_or_create_by (Class Method)

This method attempts to find a record matching the given attributes. If found, it returns that record. If not found, it creates and persists a new record with those attributes (plus any additional ones provided if the find attributes are a subset of create attributes).

  • It uses create! internally for the creation part, so it can raise exceptions if the creation fails (e.g., due to validations).


2. Reading Records

Summary of Common Finders:

  • Model.all: Retrieves all records of the model type. Returns Array(Model).

  • Model.find?(id : Pk) (or Model.find(id : Pk)): Finds a record by its primary key. Returns Model? (the instance or nil).

  • Model.find!(id : Pk): Finds a record by its primary key. Returns Model or raises DB::NoResultsError (or similar if not found).

  • Model.find_by(**attributes): Finds the first record matching the given attributes. Returns Model?.

  • Model.find_by!(**attributes): Finds the first record matching attributes. Returns Modelor raisesDB::NoResultsError.

  • Model.find_all_by(**attributes): Finds all records matching attributes. Returns Array(Model).

  • Model.first: Retrieves the first record (ordered by primary key). Returns Model?.

  • Model.last: Retrieves the last record (ordered by primary key). Returns Model?.

  • Model.count: Returns the total number of records as Int64.

  • Model.query.[condition].count: Counts records matching specific conditions.


3. Updating Records

To update existing records, you typically load an instance, modify its attributes, and then save it.

Load, Modify, and save / save!

This is the standard approach:

update / update! (Instance Methods)

These instance methods provide a shortcut to assign attributes and then save the record.

  • instance.update(attributes): Assigns the given attributes (Hash or keyword arguments) to the instance and then calls save. Returns true if successful, false otherwise.

  • instance.update!(attributes): Assigns attributes and calls save!. Raises an exception if saving fails (e.g., CQL::Errors::RecordInvalid).

Model.update!(id, attributes) (Class Method)

This class method updates a record identified by its primary key with the given attributes. It typically loads the record, updates attributes, and calls save!. Can raise DB::NoResultsError if the ID is not found, or validation/save exceptions.

Batch Updates (Model.update_by, Model.update_all)

These methods allow updating multiple records at once without instantiating each one.

  • Model.update_by(conditions_hash, updates_hash): Updates all records matching conditions_hash with the attributes in updates_hash.

  • Model.update_all(updates_hash): Updates all records in the table with the attributes in updates_hash. Use with extreme caution!

These methods typically execute a single SQL UPDATE statement and do not instantiate records, run validations, or trigger callbacks.


4. Deleting Records

Records can be deleted individually or in batches.

delete! (Instance Method)

Deletes the specific model instance from the database.

  • Runs before_destroy and after_destroy callbacks.

  • Returns true if successful. Returns false if a before_destroy callback halts the operation.

Model.delete!(id : Pk) (Class Method)

Deletes the record with the specified primary key directly from the database.

  • This method typically does not run Active Record callbacks (before_destroy, after_destroy) as it usually issues a direct SQL DELETE command.

  • It might raise an exception if the record doesn't exist or if there's a database error.

Batch Deletes (Model.delete_by!, Model.delete_all)

These class methods delete multiple records based on conditions or all records from a table.

  • Model.delete_by!(attributes_hash): Deletes all records matching the attributes_hash.

  • Model.delete_all: Deletes all records from the model's table. Use with extreme caution!

These methods typically execute direct SQL DELETE statements and do not instantiate records or run Active Record callbacks.

Always be careful with batch delete operations, especially delete_all, as they can lead to irreversible data loss if not used correctly.


CQL provides various methods to retrieve records from the database. Many of these are covered in detail in the .

For building more complex queries (e.g., with joins, specific selections, grouping), refer to the .

This guide covers the primary CRUD operations available in CQL Active Record. For more advanced querying, refer to the , and for information on lifecycle events and data integrity, see the guides on and .

AcmeDB2 = CQL::Schema.build(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]) do

  table :movies do
    primary :id, Int64, auto_increment: true
    text :title
  end

  table :screenplays do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :content
  end

  table :actors do
    primary :id, Int64, auto_increment: true
    text :name
  end

  table :directors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    text :name
  end

  table :movies_actors do
    primary :id, Int64, auto_increment: true
    bigint :movie_id
    bigint :actor_id
  end
end
MainDB = CQL::Schema.build(:main, adapter: CQL::Adapter::Postgres, uri: ENV["MAIN_DB_URL"]) do
  # Define main schema tables
end

AnalyticsDB = CQL::Schema.build(:analytics, adapter: CQL::Adapter::Postgres, uri: ENV["ANALYTICS_DB_URL"]) do
  # Define analytics schema tables
end
struct User
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :users
  property id : Int64?
  property name : String
  property email : String
  property active : Bool = false
  # ... other properties and initializers ...
end
# Using save (returns true/false)
user1 = User.new(name: "John Doe", email: "john.doe@example.com")
if user1.save
  puts "User '#{user1.name}' saved with ID: #{user1.id.not_nil!}"
else
  puts "Failed to save user '#{user1.name}': #{user1.errors.full_messages.join(", ")}"
end

# Using save! (raises on failure)
try
  user2 = User.new(name: "Jane Doe", email: "jane.doe@example.com")
  user2.save!
  puts "User '#{user2.name}' saved with ID: #{user2.id.not_nil!}"
rescue ex : CQL::Errors::RecordInvalid
  puts "Validation error for '#{user2.name}': #{ex.record.errors.full_messages.join(", ")}"
rescue ex : Exception # Catch other potential save errors
  puts "Could not save user '#{user2.name}': #{ex.message}"
end
# Using create! (raises on failure)
try
  user3 = User.create!(name: "Alice Wonderland", email: "alice@example.com", active: true)
  puts "User '#{user3.name}' created with ID: #{user3.id.not_nil!}"

  # You can also pass a hash:
  attrs = {name: "Bob The Builder", email: "bob@example.com"}
  user4 = User.create!(attrs)
  puts "User '#{user4.name}' created with ID: #{user4.id.not_nil!}"
rescue ex : CQL::Errors::RecordInvalid
  puts "Failed to create user: #{ex.record.errors.full_messages.join(", ")}"
rescue ex : Exception
  puts "Failed to create user: #{ex.message}"
end

# Using create (less common for direct use, as error handling requires checking instance.errors)
user5_attrs = {name: "Valid User", email: "valid@example.com"}
user5 = User.create(user5_attrs)
if user5 && user5.persisted?
  puts "User '#{user5.name}' created."
elsif user5 # Instance returned but not persisted
  puts "Failed to create user '#{user5.name}': #{user5.errors.full_messages.join(", ")}"
else # Create might return false directly
  puts "User creation completely halted (e.g., by a before_create callback)."
end
# Attempts to find a user by email. If not found, creates with name and email.
user_charlie = User.find_or_create_by(email: "charlie@example.com", name: "Charlie Brown")
puts "User '#{user_charlie.name}' (ID: #{user_charlie.id.not_nil!}) is present."

# If called again with the same email, it will find the existing record.
# The name attribute here would be ignored if only email is used for finding.
user_charlie_again = User.find_or_create_by(email: "charlie@example.com", name: "Charles Brown (updated attempt)")
puts "Found user '#{user_charlie_again.name}' again (ID: #{user_charlie_again.id.not_nil!}). Name should be 'Charlie Brown' unless attributes were updated separately."

# It's common to pass all necessary attributes for creation:
user_diana = User.find_or_create_by(email: "diana@example.com", name: "Diana Prince", active: true)
puts "User '#{user_diana.name}' created or found."
all_users = User.all
maybe_user = User.find?(1_i64)
# user = User.find!(1_i64)
active_admin = User.find_by(active: true, role: "admin")
# specific_user = User.find_by!(email: "jane.doe@example.com")
all_active_users = User.find_all_by(active: true)
if user_to_update = User.find_by(email: "john.doe@example.com")
  user_to_update.active = true
  user_to_update.name = "Johnathan Doe"
  # user_to_update.updated_at = Time.utc # Often handled by callbacks or DB

  if user_to_update.save
    puts "User '#{user_to_update.name}' updated."
  else
    puts "Failed to update '#{user_to_update.name}': #{user_to_update.errors.full_messages.join(", ")}"
  end
end

# Using save! for updates (raises on failure)
if user_jane = User.find_by(email: "jane.doe@example.com")
  user_jane.name = "Jane D. Updated"
  begin
    user_jane.save!
    puts "User '#{user_jane.name}' updated successfully with save!"
  rescue ex : CQL::Errors::RecordInvalid
    puts "Failed to update '#{user_jane.name}': #{ex.record.errors.full_messages.join(", ")}"
  rescue ex : Exception
    puts "Failed to update '#{user_jane.name}': #{ex.message}"
  end
end
if user_alice = User.find_by(email: "alice@example.com")
  begin
    # Using update!
    user_alice.update!(active: false, name: "Alice Inactive")
    puts "User '#{user_alice.name}' updated with instance update!"

    # Using update (returns true/false)
    if user_alice.update(name: "Alice Active Again")
      puts "User '#{user_alice.name}' updated with instance update."
    else
      puts "Failed to update Alice with instance update: #{user_alice.errors.full_messages.join(", ")}"
    end
  rescue ex : Exception
    puts "Exception during Alice's update!: #{ex.message}"
  end
end
# Assuming user_bob was created earlier and user_bob.id is available
if user_bob_id = some_user_id_variable # e.g., user_bob.id.not_nil!
  begin
    User.update!(user_bob_id, active: true, name: "Robert The Great")
    puts "User ID #{user_bob_id} updated via class update!"

    # Can also take a hash for attributes
    # User.update!(user_bob_id, {name: "Robert The Builder Again"})
  rescue ex : DB::NoResultsError
    puts "User ID #{user_bob_id} not found for update."
  rescue ex : CQL::Errors::RecordInvalid
    puts "Validation failed for User ID #{user_bob_id}: #{ex.record.errors.full_messages.join(", ")}"
  rescue ex : Exception
    puts "Failed to update User ID #{user_bob_id}: #{ex.message}"
  end
end
# Example: Make all users with email ending in '@example.com' inactive.
# Assuming a direct SQL condition is needed or `update_by` supports patterns.
# The exact capabilities of `update_by` for conditions might vary; consult CQL specifics.
# For a simple equality condition:
User.update_by({role: "guest"}, {active: false, updated_at: Time.utc})
puts "Deactivated all guest users."

# Example: Update all users to have a default status (use carefully!)
# User.update_all({status: "pending_review", updated_at: Time.utc})
# puts "Set all users to pending_review status."
if user_to_delete = User.find_by(email: "charlie@example.com")
  puts "Attempting to delete user: #{user_to_delete.name}"
  if user_to_delete.delete!
    puts "User '#{user_to_delete.name}' deleted successfully."
  else
    puts "Failed to delete user '#{user_to_delete.name}' (perhaps a before_destroy callback halted?)."
  end
else
  puts "User to delete not found."
end
if some_user = User.find_by(email: "user_to_delete_by_id@example.com")
  user_id_to_delete = some_user.id.not_nil!
  begin
    User.delete!(user_id_to_delete)
    puts "User ID #{user_id_to_delete} deleted via class delete! (callbacks likely skipped)."
  rescue ex : Exception
    puts "Failed to delete User ID #{user_id_to_delete} via class delete!: #{ex.message}"
  end
end
# Delete all users marked as inactive
User.delete_by!(active: false)
puts "Deleted all inactive users (callbacks likely skipped)."

# Delete all records from the users table (USE WITH EXTREME CAUTION!)
# User.delete_all
# puts "Deleted all users from the table (callbacks likely skipped)."
Querying Guide
Querying Guide
Querying Guide
Callbacks
Validations

Patterns

Software design patterns are general, reusable solutions to commonly occurring problems within a given context in software design. In the realm of Object-Relational Mapping (ORM) and database interaction, several established patterns help structure how applications access and manipulate data.

This section explores some of the key data access and ORM patterns that can be implemented or are relevant when working with Crystal Query Language (CQL). Understanding these patterns can help you design more maintainable, scalable, and understandable data access layers in your Crystal applications.

We will delve into the following patterns:

  • Active Record: An object that wraps a row in a database table or view, encapsulates database access, and adds domain logic on that data.

  • Entity Framework (Conceptual Overview): While CQL is not a direct port of Microsoft's Entity Framework, this section will discuss some conceptual similarities or how related ideas (like a rich object context and LINQ-style querying) might apply or inspire usage with CQL.

  • Repository: Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects.

Each pattern offers different trade-offs in terms of simplicity, flexibility, testability, and separation of concerns. Explore the specific guides to understand how they can be applied with CQL.

Entity Framework

In the context of CQL (Crystal Query Language), Entity Framework (EF) serves as a useful comparison to help developers understand how CQL and its features (like migrations, schema management, and object-relational mapping) work. Just as EF simplifies database interactions in .NET applications, CQL does the same for Crystal applications. Let’s break down the key concepts and approaches of EF and see how they align with CQL’s functionalities.

1. Development Patterns

In Entity Framework, developers have three approaches to database design: Database-First, Code-First, and Model-First. CQL shares some similarities, especially with the Code-First and Database-First approaches, but with Crystal-specific tooling.

CQL's Schema-First Approach (Similar to Code-First)

CQL primarily uses a Schema-First approach, where you define your database schema using Crystal code and CQL builds and manages the database based on this schema. This is similar to EF’s Code-First approach, where the developer defines the entity classes and EF generates the database schema.

Example in CQL:

schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
  table :users do
    primary :id, Int32
    column :name, String
    column :age, Int32
  end
end
schema.init

This code defines the users table and its columns (id, name, and age), which CQL will use to generate the corresponding database structure.

2. Core Concepts in CQL (Similar to EF Concepts)

Schema and Tables (Equivalent to EF’s DbContext and DbSet)

In EF, a DbContext is used to manage the entities (mapped to database tables), and each DbSet represents a collection of entities (mapped to individual tables). Similarly, in CQL:

  • CQL::Schema manages the structure of your database.

  • Tables are defined within the schema using table blocks.

Example:

schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
  table :products do
    primary :id, Int32
    column :name, String
    column :price, Float64
  end
end

This is similar to defining DbSet<Product> in EF, which represents the products table.

Migrations (Similar to EF Migrations)

In CQL, migrations are a key feature, similar to EF’s migrations, for managing database schema changes over time. Migrations allow you to evolve your schema, apply changes, roll them back, and maintain a history of modifications.

Example migration in CQL:

class CreateUsersTable < CQL::Migration
  self.version = 1_i64

  def up
    schema.alter :users do
      add_column :name, String
      add_column :age, Int32
    end
  end

  def down
    schema.alter :users do
      drop_column :name
      drop_column :age
    end
  end
end

In EF, migrations are handled using commands like Add-Migration and Update-Database. In CQL, migrations are applied and managed using the CQL::Migrator class, which provides methods for applying, rolling back, and listing migrations.

Applying migrations in CQL:

migrator = CQL::Migrator.new(schema)
migrator.up  # Apply all pending migrations

This is similar to EF’s Update-Database command, which applies migrations to the database.

Entities and Relationships (Similar to EF Entities and Navigation Properties)

In CQL, database tables and columns are defined as part of the schema, and relationships like foreign keys can be established. This is comparable to how entities and relationships between them are managed in EF.

Example in CQL (adding a foreign key):

schema = CQL::Schema.build(:my_db, adapter: CQL::Adapter::SQLite, uri: "sqlite3://db.sqlite3") do |s|
  table :orders do
    primary :id, Int32
    bigint :user_id
    text :order_date
    foreign_key :user_id, :users, :id, on_delete: "CASCADE"
  end
end

This defines an orders table with a foreign key relationship to the users table, similar to how EF handles one-to-many or many-to-many relationships.

3. Life Cycle of Migrations and Schema Changes

CQL’s migration life cycle aligns closely with EF’s migrations system. In both cases, you:

  1. Define migrations to introduce schema changes.

  2. Apply migrations to update the database schema.

  3. Rollback or redo migrations if needed to manage schema changes.

Example Migration Workflow in CQL:

migrator.up   # Apply pending migrations
migrator.down # Rollback the last migration
migrator.redo # Rollback and reapply the last migration

In EF, you would use commands like Update-Database, Remove-Migration, and Add-Migration to manage these operations.

4. Querying the Database (CQL vs. EF’s LINQ to Entities)

In EF, developers use LINQ to query entities and interact with the database. CQL also allows querying the database but through SQL-like syntax, aligning with Crystal’s programming style.

Example query in CQL:

AcmeDB
  .query
  .from(:products)
  .select(:name, :price)
  .where(name: "Laptop")
  .all(Product)

This retrieves all products where the name is "Laptop", similar to how LINQ queries work in EF:

var laptops = dbContext.Products.Where(p => p.Name == "Laptop").ToList();

Both frameworks provide abstractions that avoid writing raw SQL, but CQL maintains a more SQL-like approach in its query building.

5. Advantages of Using CQL for Crystal Developers

  • Productivity: Like EF, CQL reduces the need for writing raw SQL by allowing developers to work with objects (schemas, tables, columns).

  • Schema Management: CQL migrations simplify managing database changes, ensuring your schema evolves without breaking changes.

  • Consistency: CQL’s Schema-First approach ensures the database schema is in sync with your application’s Crystal code, similar to EF’s Code-First approach.


Conclusion

CQL provides similar ORM-like features for Crystal that Entity Framework does for .NET. Whether using migrations, schema definitions, or querying data, both CQL and EF streamline database interactions, making it easier to manage complex applications. By understanding the core parallels between these two systems, Crystal developers can better leverage CQL’s powerful schema management and migration tools to build scalable and maintainable database-driven applications.

Getting Started

Welcome to CQL Active Record! This guide will help you set up CQL in your Crystal project, connect to a database, define your first model, and perform basic CRUD, and first query. Use idiomatic Crystal and CQL, and provide clear, step-by-step examples.


1. Installation

Add CQL to your shard.yml:

dependencies:
  cql:
    github: your-org/cql
    version: ~> 1.0

Then install dependencies:

shards install

2. Database Setup

Configure your database connection. For example, to use PostgreSQL:

require "cql"
require "cql/pg"

# Define a database context
AcmeDB = CQL::DBContext.new(
  CQL::PG::Driver.new(
    host: "localhost",
    user: "postgres",
    password: "password",
    database: "acme_db"
  )
)

3. Defining a Model

Create a model that maps to a table:

struct User
  include CQL::ActiveRecord::Model(Int32)
  db_context AcmeDB, :users

  property id : Int32?
  property name : String
  property email : String
  property active : Bool = false
  property created_at : Time?
  property updated_at : Time?
end

4. Running Migrations

Create your tables using migrations (optional but recommended):

class CreateUsers < CQL::Migration
  self.version = 1_i64

  def up
    schema.create :users do
      primary_key :id, :serial
      column :name, :string
      column :email, :string
      column :active, :bool, default: false
      column :created_at, :timestamp
      column :updated_at, :timestamp
    end
  end

  def down
    schema.drop :users
  end
end

Run your migrations using your preferred migration runner.


5. Basic CRUD Operations

Create

user = User.new(name: "Alice", email: "alice@example.com")
user.save

Read

user = User.query.where(name: "Alice").first(User)
puts user.try(&.email)

Update

user = User.query.where(name: "Alice").first(User)
if user
  user.active = true
  user.save
end

Delete

user = User.query.where(name: "Alice").first(User)
user.try(&.delete)

6. Your First Query

# Find all active users
active_users = User.query.where(active: true).all(User)
active_users.each { |u| puts u.name }

Next Steps

  • Defining Models

  • Querying

  • Complex Queries

  • Validations, Callbacks, and More

Transactions

This guide explores how to leverage database transactions effectively within your Crystal applications using CQL's Active Record pattern. We will use a practical banking domain example throughout to illustrate key concepts and best practices.

This documentation covers the functionality provided by the @transactions.md and @transactional.cr modules within the library.

1. Introduction to Transactions in CQL

What are Transactions?

In database systems, a transaction is a single unit of work. This unit comprises one or more operations that are treated as an indivisible sequence. The core principle is that either all operations within the transaction complete successfully (commit) or none of them do (rollback).

Imagine a simple task like transferring money between two bank accounts. This isn't just one database operation; it typically involves:

  1. Debiting the sender's account.

  2. Crediting the recipient's account.

If the debit succeeds but the credit fails (e.g., due to a network error or a constraint violation), the system would be in an inconsistent state – money is gone from one account but didn't appear in the other. Transactions prevent this by ensuring that both steps must succeed together. If the credit fails, the debit is automatically undone.

Why are they important in domain modeling?

In application development, especially when dealing with complex business logic and multiple related data changes, transactions are crucial for:

  • Maintaining Data Consistency: Ensuring that your database adheres to all predefined rules and constraints.

  • Preventing Partial Updates: Avoiding scenarios where only a portion of a multi-step operation completes, leaving data in an invalid state.

  • Handling Concurrency: Providing a mechanism to manage simultaneous access and modifications to data by multiple users or processes (though concurrency requires careful consideration of isolation levels and potential deadlocks).

  • Simplifying Error Recovery: If an error occurs within a transaction, you know the database will revert to its state before the transaction began, making error handling and recovery more predictable.

Transactions provide the crucial ACID properties:

  • Atomicity: The transaction is a single unit; either it completes entirely or has no effect.

  • Consistency: A transaction brings the database from one valid state to another.

  • Isolation: Concurrent transactions do not interfere with each other. Each sees the database as if it were running alone.

  • Durability: Once a transaction is committed, the changes are permanent and survive system failures.

Supported Features in the library

CQL's Active Record implementation simplifies transaction management in Crystal. By including the CQL::ActiveRecord::Transactional module in your model, you gain access to the transaction class method. This method provides a block-based interface for executing a series of database operations within a single transaction.

Use Case: Bank Transfer

To demonstrate transactions, we will use a simplified banking application.

Overview of the domain

The core operation is transferring money between accounts. This operation must be atomic – the debit from one account and the credit to another must happen together. We also need to track these operations for auditing purposes.

Entities Involved

  • BankAccount: Stores account details and balance.

  • Transaction: Records deposits, withdrawals, and transfers.

  • AuditLog: Provides a detailed log of system activities.

(Detailed schema and model definitions are omitted for brevity, but assume standard columns like id, balance, amount, type, created_at, etc., as introduced in the schema section of previous versions).

3. Writing Transactional Logic with CQL

The core logic for banking operations like withdrawals, deposits, and transfers involves updating one or more BankAccount records, creating a Transaction record, and often creating an AuditLog record. These steps must be performed atomically. This is where the BankAccount.transaction block becomes essential.

The BankAccount.transaction Block

Any database operations performed using CQL Active Record methods inside a BankAccount.transaction do ... end block are treated as a single unit by the database.

  • When the block is entered, a database transaction is started (BEGIN).

  • When the block completes without raising an exception, the transaction is committed (COMMIT), making all changes permanent.

  • If any exception is raised within the block, the transaction is automatically rolled back (ROLLBACK), discarding all changes made during the transaction.

  • You can also explicitly call tx.rollback on the yielded transaction object tx to manually roll back.

This ensures that if any step of a multi-operation process fails, the entire set of operations is undone, maintaining data integrity.

Transaction Logic Examples

Here are the core transaction blocks for our banking operations, demonstrating the use of CQL features within the atomic unit:

Withdrawal Transaction

This transaction debits a BankAccount and records the event.

Why a Transaction is Useful Here: While a withdrawal is simpler than a transfer, using a transaction ensures that the account balance update and the recording of the transaction/audit log happen together. If the record-keeping fails, the balance change is undone.

Deposit Transaction

This transaction credits a BankAccount and records the event.

Why a Transaction is Useful Here: Similar to withdrawal, it guarantees that the balance update and the logging/recording of the deposit happen together.

Transfer Transaction (Highlighting Atomicity)

This is the prime example where transactions are critical. Money must be debited from one account and credited to another and records created, all or nothing.

Why a Transaction is Crucial Here: This is the quintessential transaction use case. The debit and credit must happen together. If the debit succeeds but the credit fails (or vice versa, or if the transaction/audit log creation fails), the transaction ensures that all changes are undone, preventing money from being lost or duplicated. It guarantees Atomicity.

Handling Failures and Rollback

As shown in the conceptual examples, any unhandled exception raised within the BankAccount.transaction do |tx| ... end block will automatically trigger a database rollback. This is a key feature that ensures atomicity.

You can also explicitly roll back the transaction using the transaction object yielded to the block:

When tx.rollback is called, the database driver is instructed to perform a rollback. Any subsequent operations within the block before it exits will also be part of the rolled-back transaction.

Querying Transaction History and Audit Logs using CQL DSL

Use CQL's query builder (.query, .where, .order, .all, .exists?) to inspect the records created by successful transactions:

Explicit Commit and Rollback

While CQL transactions automatically commit if the block finishes without an unhandled exception and automatically roll back if an exception occurs, you can also explicitly control the outcome using tx.commit and tx.rollback on the yielded transaction object.

Explicit Rollback:

You can manually trigger a rollback at any point within the transaction block. This is useful if business logic dictates that a transaction should not proceed, even if no technical error (exception) has occurred.

Explicit Commit:

Similarly, you can explicitly commit a transaction before the end of the block. This can be useful in more complex scenarios, though it's less common than explicit rollback or relying on the implicit commit.

Important Note:

As stated in the Crystal DB documentation: After commit or rollback are used, the transaction is no longer usable. The connection is still open but any statement will be performed outside the context of the terminated transaction. This means you should typically not perform further database operations relying on that specific tx object after calling tx.commit or tx.rollback.

4. Nested Transactions (Savepoints)

CQL supports nested transactions using database savepoints. This is useful for sub-operations within a larger transaction that might need to be rolled back independently without affecting the outer transaction.

To create a nested transaction, pass an existing transaction object (e.g., outer_tx) to the Model.transaction method: Model.transaction(outer_tx) do |inner_tx| ... end. This creates a SAVEPOINT.

Here are common scenarios:

Scenario 1: Successful Inner and Outer Commit

Both nested and outer operations are committed.

Scenario 2: Inner Rollback (Explicit inner_tx.rollback), Outer Commit

Only inner operations roll back. Outer operations commit.

After inner_tx.rollback, the inner_tx object is no longer usable for DB operations in that context.

Scenario 3: Inner Rollback (using raise DB::Rollback), Outer Commit

DB::Rollback in the inner transaction rolls back only inner operations. The exception is handled internally, allowing the outer transaction to commit.

Scenario 4: Inner Failure (Standard Exception), Entire Transaction Rolls Back

A standard exception in the inner block, if not caught and handled within that inner block, rolls back both inner and outer transactions.

Scenario 5: Outer Rollback After Inner Success

If the inner transaction completes but the outer transaction subsequently rolls back, all changes (inner and outer) are discarded.

Key Summary Points:

  • Savepoints: Nested transactions (Model.transaction(outer_tx)) use database SAVEPOINTs.

  • inner_tx.rollback: Rolls back only the inner transaction to its savepoint. The outer transaction can continue and commit.

  • raise DB::Rollback in Inner: Same effect as inner_tx.rollback. The DB::Rollback exception is handled by the inner transaction logic and doesn't cause the outer transaction to fail.

  • Other Exceptions in Inner: If not caught within the inner block, will roll back the inner transaction and propagate to roll back the outer transaction.

  • Outer Rollback: If the outer transaction rolls back, all work (including successful inner transactions) is undone.

  • Commit: Inner transaction changes are permanent only if the outermost transaction commits.

  • Database Driver Dependency: Behavior relies on the database and driver supporting savepoints.

Use nested transactions judiciously for sub-units needing independent rollback within a larger atomic operation, as they add complexity.

5. Best Practices

Follow these best practices when working with transactions in CQL:

  • Keep transactions short and focused: Minimize the amount of work inside a transaction block. Long-running transactions hold locks longer, increasing contention and deadlocks.

  • Validate data before entering transactions: Perform necessary validation before starting the transaction to avoid unnecessary rollbacks.

  • Encapsulate logic: Wrap complex transactional logic in dedicated methods or Service Objects for organization and testability.

  • Handle exceptions: Catch exceptions outside the block for better logging and error handling, while relying on the automatic rollback inside.

  • Ensure Consistent Resource Access Order: Access multiple records within a transaction in a consistent order (e.g., by primary key ID) to reduce deadlocks.

  • Consider Isolation Levels: Understand and potentially configure isolation levels for advanced concurrency.

  • Avoid Nested Transactions: Rely on the single outer transaction.

6. Troubleshooting and Gotchas

  • Deadlocks: Caused by transactions waiting for each other's locks. Consistent resource ordering helps.

  • Silent Transaction Failures: Ensure errors inside transactions are handled and reported properly outside the block.

  • Connection Issues: Implement retry logic for transient database connection problems.

  • Misunderstanding Rollback: Rollback affects all database operations within the transaction block.

By diligently applying these principles and patterns, you can effectively use CQL's transaction capabilities to build robust and reliable applications that maintain data integrity even in the face of errors or concurrent access.

# Include this module in your model
struct MyModel
  include CQL::ActiveRecord::Model(Int32)
  include CQL::ActiveRecord::Transactional # <--- Gives access to .transaction

  db_context MY_DB, :my_models

  # ... properties
end

# Use the transaction block
MyModel.transaction do |tx|
  # All database operations within this block form a single transaction
  # If an unhandled exception occurs, the transaction is automatically rolled back
  # You can also use tx.rollback to manually roll back
end
# Conceptual method demonstrating the core transaction logic for withdrawal
def perform_withdrawal_transaction(account : BankAccount, amount : Float64)
  # Assumes validation (amount > 0, sufficient funds) happened BEFORE this block

  BankAccount.transaction do |tx|
    # 1. Update the account balance using CQL Active Record save!
    account.balance -= amount
    account.updated_at = Time.utc
    account.save! # Persists the balance change within the transaction

    # 2. Create a transaction record using CQL Active Record create!
    Transaction.create!( # Creates a new record within the transaction
      amount: amount,
      transaction_type: "withdrawal",
      from_account_id: account.id,
      status: "completed",
      created_at: Time.utc
    )

    # 3. Create an audit log entry using CQL Active Record create!
    AuditLog.create!( # Creates a new record within the transaction
      action: "withdrawal",
      entity_type: "bank_account",
      entity_id: account.id.not_nil!,
      data: { account: account.account_number, amount: amount }.to_json,
      created_at: Time.utc
    )

    # If any of the above .save! or .create! calls fail (e.g., DB error),
    # or if an exception is raised, the transaction will roll back automatically.
  end
end
# Conceptual method demonstrating the core transaction logic for deposit
def perform_deposit_transaction(account : BankAccount, amount : Float64)
  # Assumes validation (amount > 0) happened BEFORE this block

  BankAccount.transaction do |tx|
    # 1. Update the account balance
    account.balance += amount
    account.updated_at = Time.utc
    account.save! # Persists the balance change within the transaction

    # 2. Create a transaction record
    Transaction.create!( # Creates a new record within the transaction
      amount: amount,
      transaction_type: "deposit",
      to_account_id: account.id,
      status: "completed",
      created_at: Time.utc
    )

    # 3. Create an audit log entry
    AuditLog.create!( # Creates a new record within the transaction
      action: "deposit",
      entity_type: "bank_account",
      entity_id: account.id.not_nil!,
      data: { account: account.account_number, amount: amount }.to_json,
      created_at: Time.utc
    )
  end
end
# Conceptual method demonstrating the core transaction logic for transfer
def perform_transfer_transaction(from_account : BankAccount, to_account : BankAccount, amount : Float64)
  # Assumes validation (amount > 0, different accounts, sufficient funds)
  # happened BEFORE this block.
  # Also assumes accounts are sorted by ID outside for deadlock prevention.

  BankAccount.transaction do |tx|
    # Operations within the transaction - ENSURING ATOMICITY

    # 1. Reload accounts to get freshest data under current isolation level
    # This is crucial for handling concurrent access. Use CQL Active Record reload.
    from_account.reload
    to_account.reload

    # Re-validate insufficient funds after reloading
    raise "Insufficient funds after reload" if from_account.balance < amount

    # 2. Debit the sender's account using CQL Active Record save!
    from_account.balance -= amount
    from_account.updated_at = Time.utc
    from_account.save! # Persists the debit within the transaction

    # 3. Credit the recipient's account using CQL Active Record save!
    to_account.balance += amount
    to_account.updated_at = Time.utc
    to_account.save! # Persists the credit within the transaction

    # 4. Create a transaction record using CQL Active Record create!
    Transaction.create!( # Creates a new record within the transaction
      amount: amount,
      transaction_type: "transfer",
      from_account_id: from_account.id,
      to_account_id: to_account.id,
      status: "completed",
      created_at: Time.utc
    )

    # 5. Create an audit log entry using CQL Active Record create!
    AuditLog.create!( # Creates a new record within the transaction
      action: "money_transfer",
      entity_type: "transfer",
      entity_id: nil, # No single entity ID for the transfer action
      data: { from: from_account.account_number, to: to_account.account_number, amount: amount }.to_json,
      created_at: Time.utc
    )

    # If any of these steps (reloading, saving, creating records) fails,
    # the entire transaction is rolled back, guaranteeing that no partial
    # changes are saved to the database.
  end
end
BankAccount.transaction do |tx|
  # Perform some operations...

  if some_business_condition_is_not_met
    puts "Condition not met, rolling back!"
    tx.rollback # Explicitly roll back all changes made so far in this block
    # Note: Raising an exception immediately after tx.rollback is common
    # to stop execution and indicate failure.
    raise "Manual rollback triggered"
  end

  # If no rollback or exception occurs, the transaction is committed here
end
# Get all transfers from Alice's account
alice_transfer_history = Transaction.query
  .where(from_account_id: alice_account.id)
  .order(created_at: :desc)
  .all(Transaction) # Fetch all matching records as Transaction objects

puts "\n--- Alice's Transfer History ---"
alice_transfer_history.each { |tx| puts "- $#{tx.amount} to Account ID: #{tx.to_account_id} (#{tx.status})" }

# Get all audit logs related to Bob's account
bob_account_audit_logs = AuditLog.query
  .where(entity_type: "bank_account", entity_id: bob_account.id)
  .order(created_at: :asc)
  .all(AuditLog) # Fetch all matching records as AuditLog objects

puts "\n--- Bob's Account Audit Logs ---"
bob_account_audit_logs.each { |log| puts "- #{log.action}: #{log.data}" }
BankAccount.transaction do |tx|
  from_account = BankAccount.find_by(account_number: "ACC_SENDER")
  to_account = BankAccount.find_by(account_number: "ACC_RECEIVER")
  transfer_amount = 50.0

  unless from_account && to_account
    puts "Error: One or both accounts not found."
    tx.rollback # Explicitly roll back
    # Consider raising an exception or returning early after rollback
    # to prevent further operations within this block.
    raise "Account lookup failed, transaction rolled back."
  end

  if from_account.balance < transfer_amount
    puts "Insufficient funds. Rolling back transaction."
    tx.rollback # Explicitly roll back
    raise "Insufficient funds, transaction rolled back."
  end

  # Proceed with operations if checks pass
  from_account.balance -= transfer_amount
  from_account.save!
  to_account.balance += transfer_amount
  to_account.save!

  puts "Transfer appears successful before explicit decision."
  # If we reach here, an implicit commit would happen at the end of the block.
  # However, we could also make an explicit decision.
end
BankAccount.transaction do |tx|
  account = BankAccount.create!(account_number: "ACC789", balance: 200.0)
  puts "Account ACC789 created with balance #{account.balance}"

  # Perform some critical updates
  account.balance += 100.0
  account.save!
  puts "Balance updated to #{account.balance}"

  # At this point, we decide to commit the changes immediately.
  puts "Explicitly committing transaction."
  tx.commit

  # Further operations here would be outside the just-committed transaction.
  # For example, trying to use `account.save!` again would likely start a new implicit transaction
  # or operate outside any transaction if auto-commit is on for the connection.
  # It is generally best practice to exit the block or not perform further DB
  # operations relying on this specific transaction after an explicit commit/rollback.
end
BankAccount.transaction do |outer_tx|
  account = BankAccount.create!(account_number: "ACC001", balance: 100.0)
  # Outer operation

  BankAccount.transaction(outer_tx) do |inner_tx|
    account.balance += 50.0
    account.save!
    # Inner operation: changes are staged
  end
  # Outer transaction commits: all changes (outer + inner) are now permanent.
end
# Expected: ACC001 has balance 150.0
BankAccount.transaction do |outer_tx|
  account = BankAccount.create!(account_number: "ACC002", balance: 100.0)
  # Outer operation

  BankAccount.transaction(outer_tx) do |inner_tx|
    account.balance += 50.0
    account.save! # Staged change
    inner_tx.rollback # Rolls back the +50.0
  end
  # account.reload # In-memory `account` might be stale; DB reflects rollback.
  # Outer transaction commits: only initial creation is permanent.
end
# Expected: ACC002 has balance 100.0
BankAccount.transaction do |outer_tx|
  account = BankAccount.create!(account_number: "ACC003", balance: 100.0)
  # Outer operation

  begin
    BankAccount.transaction(outer_tx) do |inner_tx|
      account.balance += 50.0
      account.save! # Staged change
      raise DB::Rollback.new # Rolls back +50.0; outer transaction continues
    end
  rescue ex # Catches other exceptions, not DB::Rollback from inner_tx.transaction
    puts "This line should not be reached by DB::Rollback: #{ex}"
  end
  # account.reload # DB reflects inner rollback.
  # Outer transaction commits.
end
# Expected: ACC003 has balance 100.0
begin
  BankAccount.transaction do |outer_tx|
    account = BankAccount.create!(account_number: "ACC004", balance: 100.0)
    # Outer operation

    BankAccount.transaction(outer_tx) do |inner_tx|
      account.balance += 50.0
      account.save! # Staged change
      raise "Inner operation failed!" # This exception will propagate
    end
    # This part is not reached
  end
rescue StandardError => e
  puts "Transaction failed: #{e.message}" # Logs "Inner operation failed!"
end
# Expected: ACC004 does not exist or transaction rolled back.
begin
  BankAccount.transaction do |outer_tx|
    account = BankAccount.create!(account_number: "ACC005", balance: 100.0)
    # Outer operation

    BankAccount.transaction(outer_tx) do |inner_tx|
      account.balance += 50.0
      account.save!
      # Inner operations successful relative to its savepoint
    end

    raise "Outer operation failed post-inner success!" # Causes full rollback
  end
rescue StandardError => e
  puts "Transaction failed: #{e.message}" # Logs "Outer operation failed post-inner success!"
end
# Expected: ACC005 does not exist or transaction rolled back.

Has One

In this guide, we'll cover the HasOne relationship using CQL's Active Record syntax. Like in the previous BelongsToguide, we'll start with an Entity-Relationship Diagram (ERD) to visually represent how the HasOne relationship works and build on the structure we already introduced with the BelongsTo relationship.

What is a HasOne Relationship?

The HasOne relationship indicates that one entity (a record) is related to exactly one other entity. For example, a User can have one Profile associated with it. This relationship is a one-to-one mapping between two entities.

Example Scenario: Users and Profiles

Let's say we have a system where:

  • A User can have one Profile.

  • A Profile belongs to one User.

We will represent this one-to-one relationship using CQL's HasOne and BelongsTo associations.


Defining the Schema

We'll define the users and profiles tables in the schema using CQL.

AcmeDB = CQL::Schema.define(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do
  table :users do
    primary
    text :name
    text :email
  end

  table :profiles do
    primary
    bigint :user_id, index: true
    text :bio
    text :avatar_url
  end
end
  • users table: Stores user details like name and email.

  • profiles table: Stores profile details like bio and avatar_url. It has a user_id foreign key referencing the userstable.


Defining the Models

Let's define the User and Profile models in CQL, establishing the HasOne and BelongsTo relationships.

User Model

struct User
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :users

  property id : Int64?
  property name : String
  property email : String

  # Initializing a new user with name and email
  def initialize(@name : String, @email : String)
  end

  # Association: A User has one Profile
  has_one :profile, Profile
end
  • The has_one :profile, Profile association in the User model indicates that each user has one profile. The foreign key (e.g., user_id) is expected on the profiles table.

Profile Model

struct Profile
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :profiles

  property id : Int64?
  property user_id : Int64?
  property bio : String
  property avatar_url : String

  # Initializing a profile. User can be associated later.
  def initialize(@bio : String, @avatar_url : String, @user_id : Int64? = nil)
  end

  # Association: A Profile belongs to one User
  belongs_to :user, User, foreign_key: :user_id
end
  • The belongs_to :user, User, foreign_key: :user_id association in the Profile model links each profile to a user.

Creating and Querying Records

Now that we have defined the User and Profile models with has_one and belongs_to relationships, let's see how to create and query records.

Creating a User and Profile

Option 1: Create User, then use create_association for Profile

# 1. Create and save the User
user = User.new(name: "John Doe", email: "john@example.com")
user.save!

# 2. Create and associate the Profile using the helper method
# This creates a Profile, sets its user_id to user.id, and saves the Profile.
profile = user.create_profile(bio: "Developer at Acme", avatar_url: "avatar_url.jpg")

puts "User #{user.name} created with ID: #{user.id.not_nil!}"
puts "Profile for user created with ID: #{profile.id.not_nil!}, Bio: #{profile.bio}, UserID: #{profile.user_id}"

Option 2: Create User, then build and save Profile manually

user2 = User.new(name: "Jane Roe", email: "jane@example.com")
user2.save!

# Build the profile (in memory, foreign key is set)
new_profile = user2.build_profile(bio: "Designer at Innovations", avatar_url: "jane_avatar.png")
# new_profile.user_id is now user2.id

# You must save the profile separately if you use build_profile
new_profile.save!

puts "User #{user2.name} created with ID: #{user2.id.not_nil!}"
puts "Built and saved Profile ID: #{new_profile.id.not_nil!}, UserID: #{new_profile.user_id}"

Option 3: Manual creation (less common with helpers available)

user3 = User.new(name: "Manual Mike", email: "mike@example.com")
user3.save!

profile_manual = Profile.new(bio: "Manual bio", avatar_url: "manual.jpg")
profile_manual.user_id = user3.id # Manually set foreign key
# or profile_manual.user = user3   # Use belongs_to setter if preferred before save
profile_manual.save!

puts "User #{user3.name} (ID: #{user3.id.not_nil!}) and Profile (ID: #{profile_manual.id.not_nil!}) created manually."

Accessing the Profile from the User

Once a user and their profile have been created, you can retrieve the profile using the has_one association.

# Fetch the user
user = User.find(1)

# Fetch the associated profile
profile = user.profile

puts profile.bio  # Outputs: "Developer at Acme"

Here, user.profile fetches the profile associated with the user.

Accessing the User from the Profile

Similarly, you can retrieve the associated user from the profile.

# Fetch the profile
profile = Profile.find(1)

# Fetch the associated user
user = profile.user

puts user.name  # Outputs: "John Doe"

In this example, profile.user fetches the User associated with that Profile.

Deleting the Profile

You can also delete the associated profile.

# Fetch the user
user_for_delete_test = User.find_by!(email: "john@example.com") # Assuming John exists

# Option 1: Using the association getter and then instance delete
if existing_profile = user_for_delete_test.profile
  existing_profile.delete!
  puts "Profile for #{user_for_delete_test.name} deleted via instance delete."
end

# Re-create for next example
user_for_delete_test.create_profile(bio: "Temporary bio", avatar_url: "temp.jpg")

# Option 2: Using the has_one delete helper (if available and user.profile exists)
if user_for_delete_test.delete_profile # This method returns true if successful
  puts "Profile for #{user_for_delete_test.name} deleted via user.delete_profile helper."
else
  puts "Could not delete profile for #{user_for_delete_test.name} via helper, or no profile existed."
end

Similarly, deleting the user will not automatically delete the associated profile unless cascade rules are explicitly set in the database or handled by before_destroy callbacks on the User model.


Summary

In this guide, we explored the has_one relationship in CQL. We:

  • Define the User and Profile tables in the schema.

  • Created corresponding models, specifying the has_one relationship in the User model and the belongs_torelationship in the Profile model.

  • Demonstrated how to create, query, update, and delete records using the has_one and belongs_to associations.

Next Steps

In the next guide, we'll extend the ERD and cover the has_many relationship, which is commonly used when one entity is associated with multiple records (e.g., a post having many comments).

Feel free to experiment with the has_one relationship by adding more fields to your models, setting up validations, or extending your schema with more complex relationships.

Defining Models

CQL Active Record models are Crystal structs that map directly to database tables. Each model encapsulates the table's columns as properties, provides type-safe access to data, and includes methods for persistence, querying, and associations.


Basic Model Definition

To define a model:

Key points:

  • Use struct for models (Crystal convention for value types).

  • include CQL::ActiveRecord::Model(PkType) mixes in all Active Record features. The type parameter specifies the primary key type (e.g., Int32, Int64, UUID).

  • db_context AcmeDB, :users links the model to a specific table in a database context.

  • Use property for each column. Nullable types (e.g., Int32?) are used for columns that may be NULL or auto-generated.


Primary Keys

  • The primary key type is specified in the Model inclusion.

  • By convention, the primary key property is named id.

  • If the primary key is auto-generated, make it nullable (id : Int32?).


Working with Attributes

Individual Attribute Access

Use the generated getter and setter methods:

Accessing All Attributes as a Hash

Use the attributes method to get a hash of all attribute values:

Mass Assignment

Set multiple attributes at once using a hash or keyword arguments:

Note: These methods only update the instance in memory. Call save to persist changes.

Attribute Names

Get all attribute names as symbols:


Best Practices

  • Use property for all columns you want to map.

  • Use nullable types for columns that may be NULL or auto-generated.

  • Use mass assignment carefully, especially with user input.

  • Use direct property access for individual attributes; use attributes for bulk operations.


Example: Complete Model


For more on querying and persistence, see the other guides in this directory.

# src/models/user.cr
require "cql"

struct User
  # Include Active Record functionality with the primary key type
  include CQL::ActiveRecord::Model(Int32)

  # Map to the 'users' table in the AcmeDB context
  db_context AcmeDB, :users

  # Define properties for each column
  property id : Int32?           # Primary key, nullable if auto-generated
  property name : String
  property email : String
  property active : Bool = false # Default value
  property created_at : Time?
  property updated_at : Time?
end
struct Product
  include CQL::ActiveRecord::Model(UUID)
  db_context StoreDB, :products

  property id : UUID?
  property name : String
  property price : Float64
end
user = User.new(name: "Alice", email: "alice@example.com")
puts user.name  # => "Alice"
user.name = "Bob"
attrs = user.attributes
# => {:id => 1, :name => "Alice", :email => "alice@example.com", ...}
puts attrs[:name]
user = User.new(name: "Placeholder", email: "placeholder@example.com")

# Using a hash
user.attributes = {name: "Jane", email: "jane@example.com", active: true}

# Using keyword arguments
user.attributes(name: "John", email: "john@example.com")
User.attribute_names # => [:id, :name, :email, :active, :created_at, :updated_at]
struct Article
  include CQL::ActiveRecord::Model(Int64)
  db_context BlogDB, :articles

  property id : Int64?
  property title : String
  property body : String
  property published : Bool = false
  property author_id : Int32
  property created_at : Time?
  property updated_at : Time?
end

Guides

Welcome to the CQL Guides section! Here you'll find practical, focused documentation to help you use CQL (Crystal Query Language) effectively in your Crystal projects.

What You'll Find

  • Active Record with CQL: Learn how to define models, perform CRUD operations, build queries, manage relationships, and ensure data integrity using CQL's Active Record pattern.

  • Querying: Explore the powerful query interface, including chainable queries, aggregations, and advanced filtering.

  • Migrations: Manage your database schema changes over time with CQL's migration system.

  • Relationships: Understand and implement associations like belongs_to, has_one, has_many, and many_to_many.

  • Other Topics: Find guides on pagination, scopes, callbacks, validations, and more.

Each guide is organized to be self-contained and practical, with code examples and explanations tailored for Crystal developers.

How to Use These Guides

  • Start with Active Record with CQL for a comprehensive overview and links to all sub-guides.

  • Use the sidebar or the links within each guide to navigate to specific topics.

  • Refer to the Getting Started guide if you're new to CQL.

Whether you're building your first Crystal app with CQL or looking to master advanced features, these guides are your go-to resource.

Updating Records

The CQL::Update class in the CQL (Crystal Query Language) module is designed to represent and execute SQL UPDATE statements in a clean and structured manner. This guide will walk you through using the class to update records in a database, providing real-world examples and detailed explanations for each method.


Key Features

  1. Update records in a database with a simple and readable syntax.

  2. Set column values dynamically using hashes or keyword arguments.

  3. Filter records with flexible WHERE conditions.

  4. Return updated columns after executing the query.

  5. Chainable methods for building complex queries effortlessly.


Real-World Example: Updating a User's Data

Let’s start with a simple example of updating a user’s name and age in the users table.

update = CQL::Update.new(schema)
  .table(:users)
  .set(name: "John", age: 30)
  .where { |w| w.id == 1 }
  .commit

This example updates the user with id = 1 to have the name "John" and age 30.


Core Methods

Below is a detailed breakdown of the key methods in the CQL::Update class and how to use them.

1. table(table : Symbol)

Purpose: Specifies the table to update.

  • Parameters: table — A symbol representing the table name.

  • Returns: Update object (for chaining).

Real-World Example: Setting the Target Table

update.table(:users)

This sets the users table as the target for the update operation.


2. set(setters : Hash(Symbol, DB::Any))

Purpose: Specifies the column values to update using a hash.

  • Parameters: setters — A hash where keys are column names and values are the new values for those columns.

  • Returns: Update object (for chaining).

Real-World Example: Updating Multiple Columns

update
  .table(:users)
  .set(name: "John", age: 30)

This sets the name and age columns to new values for the target record(s).


3. set(**fields)

Purpose: Specifies the column values to update using keyword arguments.

  • Parameters: fields — Column-value pairs as keyword arguments.

  • Returns: Update object (for chaining).

Real-World Example: Using Keyword Arguments

update
  .table(:users)
  .set(name: "Alice", active: true)

This sets the name to "Alice" and active to true.


4. where(**fields)

Purpose: Adds a WHERE clause to filter the records to be updated.

  • Parameters: fields — A hash where keys are column names and values are the conditions to match.

  • Returns: Update object (for chaining).

Real-World Example: Filtering by a Condition

update
  .table(:users)
  .set(name: "John", age: 30)
  .where(id: 1)

This adds a condition to only update the user where id = 1.


5. where(&block)

Purpose: Adds a WHERE clause using a block for more complex conditions.

  • Parameters: Block that db_contexts the condition using a filter builder.

  • Returns: Update object (for chaining).

Real-World Example: Using a Block for Conditions

update
  .table(:users)
  .set(name: "John")
  .where { |w| w.id == 1 && w.active == true }

This example updates the user where both id = 1 and active = true.


6. commit

Purpose: Executes the UPDATE query and commits the changes to the database.

  • Returns: A DB::Result object, which represents the result of the query execution.

Real-World Example: Committing the Update

update = CQL::Update.new(schema)
  .table(:users)
  .set(name: "John", age: 30)
  .where { |w| w.id == 1 }
  .commit

This commits the changes to the users table, updating the user with id = 1.


7. back(*columns : Symbol)

Purpose: Specifies the columns to return after the update.

  • Parameters: columns — An array of symbols representing the columns to return.

  • Returns: Update object (for chaining).

Real-World Example: Returning Updated Columns

update
  .table(:users)
  .set(name: "John", age: 30)
  .where(id: 1)
  .back(:name, :age)
  .commit

This will return the updated name and age columns after the update.


8. to_sql(gen = @schema.gen)

Purpose: Generates the SQL query and the parameters required for the UPDATE statement.

  • Parameters: gen — The generator used for SQL generation (default: schema generator).

  • Returns: A tuple containing the SQL query string and the parameters.

Real-World Example: Generating SQL for an Update

update = CQL::Update.new(schema)
  .table(:users)
  .set(name: "John", age: 30)
  .where(id: 1)

sql, params = update.to_sql
puts sql     # "UPDATE users SET name = $1, age = $2 WHERE id = $3"
puts params  # ["John", 30, 1]

This generates the raw SQL query and its associated parameters without executing it.


Putting It All Together

Let’s combine multiple methods to handle a more advanced use case. Suppose you want to update a user's data, but only if they are active, and you want to return their updated email address afterward:

update = CQL::Update.new(schema)

result = update
  .table(:users)
  .set(name: "Charlie", email: "charlie@example.com")
  .where { |w| w.id == 1 && w.active == true }
  .back(:email)
  .commit

puts result  # This will return the updated email address of the user.

In this query:

  • We specify the users table.

  • We update both the name and email of the user.

  • We filter the update to only apply to the active user with id = 1.

  • We return the updated email after the update is committed.


Conclusion

The CQL::Update class provides a simple yet powerful interface for building and executing UPDATE queries in a Crystal application. With chainable methods for setting values, applying conditions, and controlling the output, you can easily handle any update operation.

Whether you are updating single records or large batches, the flexibility of CQL::Update ensures that your queries remain clean, maintainable, and efficient.

Has Many

CQL Active Record: HasMany Relationship Guide

In this guide, we'll focus on the HasMany relationship using CQL's Active Record syntax. This describes a one-to-many connection between models.

What is a HasMany Relationship?

The HasMany relationship indicates that one entity (a record) is related to multiple other entities. For example, a Post can have many Comments. This relationship is a one-to-many mapping between two entities.

Example Scenario: Posts and Comments

In a blogging system:

  • A Post can have many Comments.

  • Each Comment belongs to one Post.

This is a common one-to-many relationship where one post can have multiple comments, but each comment refers to only one post.


Defining the Schema

We'll define the posts and comments tables in the schema using CQL's DSL.

  • posts table: Stores post details like title, body, and published_at.

  • comments table: Stores comment details with a foreign key post_id that references the posts table.


Defining the Models

Let's db_context the Post and Comment models and establish the HasMany and BelongsTo relationships in CQL.

Post Model

  • The has_many :comments, Comment, foreign_key: :post_id association in the Post model defines that each post can have multiple comments. The comments table must have a post_id column.

Comment Model

  • The belongs_to :post, Post, foreign_key: :post_id in the Comment model links each comment back to its post.

Working with the HasMany Collection

When you access a has_many association (e.g., post.comments), you get a collection proxy object that offers several methods to query and manipulate the associated records.

Creating and Adding Records

Retrieving Records from the Collection

The collection is enumerable and provides methods to access its records.

Reloading the Collection

If the database might have changed, reload the collection:

The has_many macro generates reload_{{association_name}} (e.g., reload_comments).

Removing and Deleting Records

Deleting a specific comment from the association (and database):

Note: The delete method on the collection typically removes the record from the database.

Clearing the association (deletes all associated comments):

  • clear usually implies deleting the associated records from the database. Be cautious with this method.

If you delete the parent record (post.delete), associated comments are not automatically deleted unless cascade: true was specified in the has_many definition or database-level cascade rules are in place.


Eager Loading

To avoid N+1 query problems when loading many posts and their comments, use eager loading:

  • includes(:comments) tells CQL to fetch all comments for the retrieved posts in a separate, optimized query.


Summary

In this guide, we've explored the HasMany relationship in CQL. We covered:

  • Defining Post and Comment models with has_many and belongs_to associations, including the foreign_key option.

  • Interacting with the has_many collection using methods like create, build, <<, all, find_by, exists?, size, delete, and clear.

  • Eager loading associations with includes.

Next Steps

In the next guide, we'll build upon this ERD and cover the ManyToMany relationship.

AcmeDB = CQL::Schema.define(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do
  table :posts do
    primary :id, Int64, auto_increment: true
    text :title
    text :body
    timestamp :published_at
  end

  table :comments do
    primary :id, Int64, auto_increment: true
    bigint :post_id
    text :body
  end
end
struct Post
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :posts

  property id : Int64?
  property title : String
  property body : String
  property published_at : Time?

  # Initializing a new post
  def initialize(@title : String, @body : String, @published_at : Time? = Time.utc)
  end

  # Association: A Post has many Comments
  # The `foreign_key` option specifies the column on the `comments` table
  # that references the `posts` table.
  has_many :comments, Comment, foreign_key: :post_id
end
struct Comment
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :comments

  property id : Int64?
  property post_id : Int64?
  property body : String

  # Initializing a comment. Post can be associated later or by the collection.
  def initialize(@body : String, @post_id : Int64? = nil)
  end

  # Association: A Comment belongs to one Post
  belongs_to :post, Post, foreign_key: :post_id
end
# Fetch or create a Post
post = Post.find_or_create_by(title: "HasMany Guide Post", body: "Content for HasMany.")
post.save! if !post.persisted? # Ensure post is saved and has an ID

# Option 1: Using `create` on the collection (builds, sets FK, saves, adds to collection)
comment1 = post.comments.create(body: "First comment via create!")
puts "Comment 1: #{comment1.body}, Post ID: #{comment1.post_id}"

# Option 2: Using `build` on the collection (builds, sets FK, but does NOT save yet)
comment2 = post.comments.build(body: "Second comment via build.")
# comment2 is now associated with post (comment2.post_id == post.id) but not saved.
puts "Comment 2 (unsaved): #{comment2.body}, Post ID: #{comment2.post_id}"
comment2.save!
puts "Comment 2 (saved): #{comment2.id}"

# Option 3: Using `<<` operator (creates a new record from an instance and saves it)
# The Comment instance should ideally not have post_id set if `<<` handles it,
# or it should match the parent post. Behavior may vary; `create` is often clearer.
comment3 = Comment.new(body: "Third comment via << operator.")
post.comments << comment3 # This will save comment3 and associate it.
puts "Comment 3: #{comment3.body}, ID: #{comment3.id}, Post ID: #{comment3.post_id}"

# Option 4: Manual creation and association (less common for adding to existing parent)
# comment4 = Comment.new(body: "Fourth comment, manual.", post_id: post.id.not_nil!)
# comment4.save!
# post.comments.reload # Important to see it in the cached collection
# Fetch the post again to ensure a clean comments collection or use post.comments.reload
loaded_post = Post.find!(post.id.not_nil!)

puts "\nComments for Post: '#{loaded_post.title}'"
# Iterating through the collection (implicitly loads if not already loaded)
loaded_post.comments.each do |comment|
  puts "- #{comment.body} (ID: #{comment.id})"
end

# Get all comments as an array
all_comments_array = loaded_post.comments.all
puts "Total comments in array: #{all_comments_array.size}"

# Find a specific comment within the association
found_comment = loaded_post.comments.find_by(body: "First comment via create!")
if c = found_comment
  puts "Found comment by body: #{c.id}"
end

# Check for existence
is_present = loaded_post.comments.exists?(body: "Second comment via build.")
puts "Does 'Second comment via build.' exist? #{is_present}"

# Get size and check if empty
puts "Number of comments: #{loaded_post.comments.size}"
puts "Are there any comments? #{!loaded_post.comments.empty?}"

# Get first comment
first_comment = loaded_post.comments.first
puts "First comment body: #{first_comment.try(&.body)}"

# Get array of IDs
comment_ids = loaded_post.comments.ids
puts "Comment IDs: #{comment_ids}"
loaded_post.comments.reload # Fetches fresh data from DB
# or for a specific association on a model instance:
# loaded_post.reload_comments # if such a specific reloader is generated by has_many macro
comment_to_delete = loaded_post.comments.find_by(body: "Third comment via << operator.")
if ctd = comment_to_delete
  if loaded_post.comments.delete(ctd) # Pass instance or its ID
    puts "Successfully deleted comment ID: #{ctd.id}"
  else
    puts "Could not delete comment ID: #{ctd.id}"
  end
end
# Verify deletion
puts "Comment count after delete: #{loaded_post.comments.size}"
# First, add some comments if cleared previously
loaded_post.comments.create(body: "Temp comment 1 for clear test")
loaded_post.comments.create(body: "Temp comment 2 for clear test")
puts "Comments before clear: #{loaded_post.comments.size}"

loaded_post.comments.clear # Deletes all comments associated with this post from the database
puts "Comments after clear: #{loaded_post.comments.size}" # Should be 0
# Fetches all posts and their associated comments in a more optimized way (typically 2 queries)
posts_with_comments = Post.query.includes(:comments).all(Post)

posts_with_comments.each do |p|
  puts "Post: #{p.title} has #{p.comments.size} comments (already loaded):"
  p.comments.each do |c| # Accesses the already loaded comments
    puts "  - #{c.body}"
  end
end

Querying

The Queryable module in CQL provides a powerful, type-safe, and chainable interface for constructing SQL queries using idiomatic Crystal code. It enables you to filter, sort, join, and aggregate records with a fluent API, abstracting away raw SQL while retaining full expressiveness and performance.

This guide covers how to use the query interface provided by CQL Active Record models and repositories, including all major query methods and best practices.


Basic Usage

You can build queries directly on your model or repository. Most query methods return a chainable query object, allowing you to compose complex queries step by step.

Example: Simple Filtering

# Find all users named "Alice"
alice_users = User.where(name: "Alice").all

# Find the first user with a specific email
user = User.where(email: "alice@example.com").first

Example: Using a Repository

user_repo = UserRepository.new
# Find all users with a given domain
gmail_users = user_repo.query.where { email.like("%@gmail.com") }.all(User)

Chainable Query Methods

CQL provides a rich set of chainable query methods that allow you to build complex queries in a type-safe and expressive way. Each method returns a query object that can be further chained with additional methods.

where

The where method is used to filter records based on conditions. It supports both simple key-value pairs and block-based conditions.

# Simple key-value conditions
User.where(name: "Alice", active: true)

# Block-based conditions with operators
User.where { age > 18 }
User.where { (name.like("A%")) & (age >= 21) }

# Multiple conditions
User.where(name: "Alice").where(active: true)

order

Sort records using the order method. You can specify multiple columns and sort directions.

# Single column ordering
User.order(:name)                    # ASC by default
User.order(name: :desc)              # Explicit DESC

# Multiple columns
User.order(:created_at, name: :desc)

# Using block syntax
User.order { created_at.desc }

limit and offset

Control the number of records returned and paginate through results.

# Get first 10 records
User.limit(10)

# Skip first 20 records and get next 10
User.offset(20).limit(10)

# Common pagination pattern
page = 2
per_page = 10
User.offset((page - 1) * per_page).limit(per_page)

select

Specify which columns to retrieve from the database.

# Select specific columns
User.select(:id, :name, :email)

# Using block syntax
User.select { [id, name, email] }

# With aliases
User.select { [id.as("user_id"), name.as("user_name")] }

group_by

Group records by one or more columns, often used with aggregate functions.

# Group by single column
User.group_by(:role)

# Multiple columns
User.group_by(:role, :status)

# With block syntax
User.group_by { [role, status] }

join

Perform SQL joins between tables. CQL supports various join types and provides a type-safe way to specify join conditions.

# Simple inner join
User.join(Post)

# Join with conditions
User.join(Post) { users.id == posts.user_id }

# Multiple joins
User.join(Post)
    .join(Comment) { posts.id == comments.post_id }

# Different join types
User.left_join(Post)
User.right_join(Post)
User.full_join(Post)

having

Add conditions to grouped records, typically used with group_by.

User.group_by(:role)
    .having { count(id) > 5 }

distinct

Remove duplicate records from the result set.

User.distinct
User.distinct(:name, :email)

Terminal Operations

Terminal operations execute the query and return the results. These methods are typically used at the end of a query chain to fetch the data from the database.

all

Returns all records matching the query as an array.

# Get all users
users = User.all

# Get all active users ordered by name
active_users = User.where(active: true)
                  .order(:name)
                  .all

# Get all users with their posts
users_with_posts = User.join(Post)
                      .select { [users.*, posts.*] }
                      .all

first and first!

Returns the first record matching the query. first returns nil if no record is found, while first! raises an error.

# Get the first user (returns nil if none found)
first_user = User.first

# Get the first user (raises error if none found)
first_user = User.first!

# Get the first active user ordered by creation date
oldest_active = User.where(active: true)
                   .order(:created_at)
                   .first

find and find!

Find a record by its primary key. find returns nil if not found, while find! raises an error.

# Find user by ID (returns nil if not found)
user = User.find(1)

# Find user by ID (raises error if not found)
user = User.find!(1)

# Find with additional conditions
active_user = User.where(active: true).find(1)

count

Returns the number of records matching the query.

# Count all users
total_users = User.count

# Count active users
active_count = User.where(active: true).count

# Count users by role
role_counts = User.group_by(:role)
                 .select { [role, count(id).as("count")] }
                 .all

exists?

Checks if any records match the query.

# Check if any users exist
has_users = User.exists?

# Check if any active users exist
has_active = User.where(active: true).exists?

# Check if a specific user exists
user_exists = User.where(email: "alice@example.com").exists?

pluck

Returns an array of values for the specified columns.

# Get all user names
names = User.pluck(:name)

# Get multiple columns
name_emails = User.pluck(:name, :email)

# With conditions
active_names = User.where(active: true)
                  .pluck(:name)

sum, average, minimum, maximum

Aggregate functions that can be used to calculate statistics.

# Calculate total age of all users
total_age = User.sum(:age)

# Calculate average age
avg_age = User.average(:age)

# Find minimum and maximum ages
min_age = User.minimum(:age)
max_age = User.maximum(:age)

# With conditions
active_avg_age = User.where(active: true)
                    .average(:age)

find_each and find_in_batches

Process records in batches to handle large datasets efficiently.

# Process users one at a time
User.find_each(batch_size: 1000) do |user|
  # Process each user
  process_user(user)
end

# Process users in batches
User.find_in_batches(batch_size: 1000) do |users|
  # Process batch of users
  process_users(users)
end

Composing Complex Queries

CQL's query interface allows you to build complex queries by combining multiple query methods. This section demonstrates common patterns and best practices for composing sophisticated queries.

Advanced Filtering

Combine multiple conditions using logical operators and complex expressions.

# Complex where conditions
User.where { (age >= 18) & (active == true) }
    .where { (name.like("A%")) | (email.like("%@gmail.com")) }

# Nested conditions
User.where { (role == "admin") & ((status == "active") | (status == "pending")) }

# Using IN clauses
User.where { role.in(["admin", "moderator"]) }
    .where { status.in(["active", "pending"]) }

Joins with Conditions

Combine joins with filtering and ordering.

# Join with filtering on both tables
User.join(Post) { users.id == posts.user_id }
    .where { (users.active == true) & (posts.published == true) }
    .order { [users.name, posts.created_at.desc] }

# Multiple joins with conditions
User.join(Post) { users.id == posts.user_id }
    .join(Comment) { posts.id == comments.post_id }
    .where { comments.created_at > 1.week.ago }
    .select { [users.*, posts.title, count(comments.id).as("comment_count")] }
    .group_by { [users.id, posts.id] }

Aggregations with Grouping

Combine grouping with aggregations and having clauses.

# User statistics by role
User.group_by(:role)
    .select { [role, count(id).as("total"),
              average(age).as("avg_age"),
              maximum(age).as("max_age")] }
    .having { count(id) > 5 }
    .order { count(id).desc }

# Post statistics with user information
User.join(Post) { users.id == posts.user_id }
    .group_by { [users.id, users.name] }
    .select { [users.name,
              count(posts.id).as("post_count"),
              average(posts.view_count).as("avg_views")] }
    .having { count(posts.id) >= 3 }

Pagination with Complex Queries

Combine pagination with other query methods.

# Paginated search with sorting
def search_users(query, page = 1, per_page = 20)
  User.where { (name.like("%#{query}%")) |
              (email.like("%#{query}%")) }
      .order(:name)
      .offset((page - 1) * per_page)
      .limit(per_page)
end

# Paginated join with conditions
def recent_user_posts(user_id, page = 1, per_page = 10)
  User.join(Post) { users.id == posts.user_id }
      .where(users.id: user_id)
      .where { posts.created_at > 1.month.ago }
      .order { posts.created_at.desc }
      .offset((page - 1) * per_page)
      .limit(per_page)
end

Subqueries

Use subqueries for complex filtering and data retrieval.

# Users with more than 5 posts
User.where { id.in(
  Post.select(:user_id)
      .group_by(:user_id)
      .having { count(id) > 5 }
) }

# Users with recent activity
User.where { id.in(
  Post.select(:user_id)
      .where { created_at > 1.week.ago }
      .union(
        Comment.select(:user_id)
               .where { created_at > 1.week.ago }
      )
) }

Common Table Expressions (CTEs)

Use CTEs for complex queries that need to reference the same subquery multiple times.

# Users with their post and comment counts
User.with(:user_stats) {
  User.select { [
    id,
    count(posts.id).as("post_count"),
    count(comments.id).as("comment_count")
  ] }
  .left_join(Post) { users.id == posts.user_id }
  .left_join(Comment) { users.id == comments.user_id }
  .group_by(:id)
}
.select { [users.*, user_stats.*] }
.join(:user_stats) { users.id == user_stats.id }

Query Scopes

Define reusable query scopes for common query patterns.

struct User
  include CQL::ActiveRecord::Model(Int32)

  # ... other code ...

  def self.active
    where(active: true)
  end

  def self.admins
    where(role: "admin")
  end

  def self.recently_created
    where { created_at > 1.week.ago }
  end

  def self.with_posts
    join(Post) { users.id == posts.user_id }
  end
end

# Using scopes
User.active
    .admins
    .recently_created
    .with_posts
    .order(:name)

Best Practices

Following these best practices will help you write more efficient, maintainable, and performant queries in CQL.

Query Performance

  1. Select Only Needed Columns

    # Instead of
    User.all
    
    # Use
    User.select(:id, :name, :email).all
  2. Use Appropriate Indexes

    • Add indexes for frequently queried columns

    • Index foreign keys and columns used in WHERE clauses

    • Consider composite indexes for commonly combined conditions

  3. Avoid N+1 Queries

    # Instead of
    users = User.all
    users.each do |user|
      posts = user.posts.all  # N+1 queries!
    end
    
    # Use
    User.join(Post)
        .select { [users.*, posts.*] }
        .all
  4. Use Batch Processing for Large Datasets

    # Instead of
    User.all.each do |user|
      process_user(user)
    end
    
    # Use
    User.find_each(batch_size: 1000) do |user|
      process_user(user)
    end

Code Organization

  1. Use Query Scopes for Reusable Logic

    struct User
      include CQL::ActiveRecord::Model(Int32)
    
      def self.active
        where(active: true)
      end
    
      def self.recent
        where { created_at > 1.week.ago }
      end
    end
    
    # Usage
    User.active.recent
  2. Extract Complex Queries to Methods

    struct User
      def self.search_by_name_or_email(query)
        where { (name.like("%#{query}%")) |
                (email.like("%#{query}%")) }
      end
    
      def self.with_post_counts
        left_join(Post) { users.id == posts.user_id }
          .group_by(:id)
          .select { [users.*, count(posts.id).as("post_count")] }
      end
    end
  3. Use Meaningful Names for Complex Queries

    def find_active_users_with_recent_posts
      User.active
          .join(Post) { users.id == posts.user_id }
          .where { posts.created_at > 1.week.ago }
          .distinct
    end

Query Safety

  1. Use Parameterized Queries

    # Instead of
    User.where { name == "#{user_input}" }
    
    # Use
    User.where(name: user_input)
  2. Validate Input Before Querying

    def search_users(query)
      return User.none if query.blank?
      User.where { name.like("%#{query}%") }
    end
  3. Handle Edge Cases

    def find_user_by_email(email)
      return nil if email.blank?
      User.where(email: email.downcase.strip).first
    end

Testing

  1. Test Query Scopes

    describe User do
      it "finds active users" do
        active_user = User.create(active: true)
        inactive_user = User.create(active: false)
    
        expect(User.active).to contain(active_user)
        expect(User.active).not_to contain(inactive_user)
      end
    end
  2. Test Complex Queries

    describe User do
      it "finds users with recent posts" do
        user = User.create
        old_post = Post.create(user: user, created_at: 2.weeks.ago)
        recent_post = Post.create(user: user, created_at: 3.days.ago)
    
        result = User.with_recent_posts
        expect(result).to contain(user)
      end
    end

Debugging

  1. Use to_sql for Query Inspection

    query = User.where(active: true).order(:name)
    puts query.to_sql  # Prints the generated SQL
  2. Log Slow Queries

    def self.log_slow_queries
      start_time = Time.monotonic
      result = yield
      duration = Time.monotonic - start_time
    
      if duration > 1.second
        Log.warn { "Slow query detected: #{duration.total_seconds}s" }
      end
    
      result
    end

Next Steps

  • Reference Table

Pagination

CQL's Active Record provides basic pagination functionality directly on your models, allowing you to easily retrieve records in chunks or pages. This is essential for handling large datasets efficiently, especially in web applications.

The pagination methods are available as class methods on any model that includes CQL::ActiveRecord::Model(Pk).


page(page_number, per_page \\= 10)

The page method is the primary way to retrieve a specific page of records.

  • page_number : Int32: The desired page number (1-indexed).

  • per_page : Int32: The number of records to include on each page. Defaults to 10.

It calculates the necessary offset and applies a limit to the query.

Example:

How it works internally: The method essentially performs:query.limit(per_page).offset((page_number - 1) * per_page).all(ModelName)


per_page(num_records)

The per_page method, when used as a standalone class method, sets a limit on the number of records returned. It effectively retrieves the first page of records with the specified num_records count.

  • num_records : Int32: The number of records to retrieve.

Example:

How it works internally: The method performs:query.limit(num_records).all(ModelName)

Note on per_page: While per_page can be called directly on the model class, its name might suggest it's primarily a modifier for other pagination logic (which isn't directly supported by chaining these specific class methods). In most common pagination scenarios, the page(page_number, per_page) method is more comprehensive as it handles both the page number and the items per page.

Using per_page(n) is equivalent to page(1, per_page: n).


Combining with Other Queries

Pagination methods are applied to the model's default query scope. If you need to paginate a filtered set of records, you would typically chain pagination methods onto a CQL::Query object obtained via YourModel.query:

The standalone User.page and User.per_page methods are convenient for simple, direct pagination on an entire table. For more complex scenarios, building the query and then applying limit and offset manually (as shown above) provides greater flexibility, and is what User.page does internally.

It's important to note that the Pagination module in active_record/pagination.cr provides these as class methods directly on the model. If you need to paginate a more complex query chain, you'd apply .limit() and .offset() to the query object itself.

# Assuming you have a User model defined:
struct User
  includes CQL::ActiveRecord::Model(Int64)
  db_context YourDB, :users
  # ... properties ...
end

# Get the first page, 10 users per page (default per_page)
page1_users = User.page(1)

# Get the second page, 10 users per page
page2_users = User.page(2)

# Get the third page, with 5 users per page
page3_users_custom_per_page = User.page(3, per_page: 5)

puts "Page 1 Users (#{page1_users.size} users):"
page1_users.each do |user|
  puts "- ID: #{user.id}, Name: #{user.name}"
end

puts "\\nPage 3 Users, 5 per page (#{page3_users_custom_per_page.size} users):"
page3_users_custom_per_page.each do |user|
  puts "- ID: #{user.id}, Name: #{user.name}"
end
# Get the first 5 users
first_5_users = User.per_page(5)

puts "\\nFirst 5 Users (#{first_5_users.size} users):"
first_5_users.each do |user|
  puts "- ID: #{user.id}, Name: #{user.name}"
end
# Get page 2 of active users, 5 per page
active_users_page2 = User.query
  .where(active: true)
  .order(created_at: :desc)
  .limit(5)  # This is per_page
  .offset((2 - 1) * 5) # This is (page_number - 1) * per_page
  .all(User)

puts "\\nActive Users, Page 2, 5 per page (#{active_users_page2.size} users):"
active_users_page2.each do |user|
  puts "- ID: #{user.id}, Name: #{user.name}, Active: #{user.active}"
end

Persistence Details

Beyond basic CRUD operations, CQL Active Record models offer methods to understand and manage their persistence state.

This guide assumes you have a model defined, for example:

struct User
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :users
  property id : Int64?
  # ... other properties ...
end

Checking Persistence Status

It's often useful to know if a model instance represents a record that already exists in the database or if it's a new record that hasn't been saved yet.

persisted?

The persisted? instance method returns true if the record is considered to be saved in the database, and false otherwise. Typically, this means the instance has a non-nil primary key (id).

# New, unsaved record
new_user = User.new(name: "Temp User", email: "temp@example.com")
puts "Is new_user persisted? #{new_user.persisted?}" # => false
puts "new_user ID: #{new_user.id.inspect}" # => nil (assuming id is Int64? and auto-generated)

# Save the record
if new_user.save
  puts "User saved."
  puts "Is new_user persisted now? #{new_user.persisted?}" # => true
  puts "new_user ID after save: #{new_user.id.inspect}"    # => 1 (or some other generated ID)
else
  puts "Failed to save new_user."
end

# Loaded record
existing_user = User.find?(new_user.id.not_nil!) # Find the user we just saved
if existing_user
  puts "Is existing_user persisted? #{existing_user.persisted?}" # => true
  puts "existing_user ID: #{existing_user.id.inspect}"
end

Use persisted? to:

  • Differentiate between new and existing records in forms or views.

  • Decide whether an update or an insert operation is appropriate in more manual scenarios.

  • Conditionally execute logic based on whether a record is already in the database.

new_record? (Conceptual / Alias for !persisted?)

Some ORMs provide a new_record? method, which is typically the opposite of persisted?. While CQL's core Persistence module might not explicitly define new_record?, you can achieve the same by checking !instance.persisted?.

user = User.new(name: "Another Temp", email: "another@example.com")
if !user.persisted?
  puts "This is a new record (not persisted)."
end

Reloading Records

Sometimes, the data for a record in your application might become stale if the corresponding row in the database has been changed by another process or a different part of your application. The reload! method allows you to refresh an instance's attributes from the database.

reload!

The reload! instance method fetches the latest data from the database for the current record (identified by its primary key) and updates the instance's attributes in place.

  • If the record no longer exists in the database (e.g., it was deleted by another process), reload! will typically raise a DB::NoResultsError or a similar RecordNotFound exception.

# Assume user_jane exists and her email is "jane.doe@example.com"
user_jane = User.find_by!(email: "jane.doe@example.com")
puts "Initial name for Jane: #{user_jane.name}"

# Simulate an external update to the database for this user
# (In a real scenario, this would happen outside this code flow)
# For example, directly via SQL: UPDATE users SET name = 'Jane Updated Externally' WHERE id = user_jane.id;

# Now, reload user_jane to get the latest data
try
  user_jane.reload!
  puts "Reloaded name for Jane: #{user_jane.name}" # Should show "Jane Updated Externally"

  # If user_jane was deleted externally before reload!
  # user_jane.reload! # This would raise DB::NoResultsError
rescue DB::NoResultsError
  puts "Record for user '#{user_jane.name}' (ID: #{user_jane.id}) no longer exists in the database."
rescue ex : Exception
  puts "An error occurred during reload: #{ex.message}"
end

Use reload! when:

  • You need to ensure you are working with the most up-to-date version of a record, especially before performing critical operations or displaying sensitive data.

  • You suspect an instance's state might be out of sync with the database due to concurrent operations.


Understanding these persistence details helps in managing the lifecycle and state of your Active Record model instances effectively.

Troubleshooting

Here are some common issues you might encounter while using CQL and how to resolve them.

Issue: NoMethodError when querying

Solution:

Ensure that your table and columns are correctly defined in the schema. For example:

table :users do
  primary :id, Int64
  column :name, String
end

If you're querying a column that doesn't exist, CQL will raise a NoMethodError.

Issue: Transaction not rolling back

Solution:

Ensure that any errors raised inside the transaction block are properly handled. If an error occurs, the transaction will be rolled back automatically.

Belongs To

In this guide, we'll cover the BelongsTo relationship using CQL's Active Record syntax. We'll start with an Entity-Relationship Diagram (ERD) to illustrate how this relationship works and continuously build upon this diagram as we introduce new relationships in subsequent guides.

What is a BelongsTo Relationship?

The BelongsTo association in a database indicates that one entity (a record) refers to another entity by holding a foreign key to that record. For example, a Comment belongs to a Post, and each comment references the Post it is associated with by storing the post_id as a foreign key.

Example Scenario: Posts and Comments

Let's say you have a blog system where:

  • A Post can have many Comments.

  • A Comment belongs to one Post.

We'll start by implementing the BelongsTo relationship from the Comment to the Post.


Defining the Schema

We'll first define the posts and comments tables using CQL's schema DSL.

codeAcmeDB = CQL::Schema.define(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do

  table :posts do
    primary
    text :title
    text :body
    timestamp :published_at
  end

  table :comments do
    primary
    bigint :post_id
    text :body
  end
end
  • posts table: Contains the blog post data (title, body, and published date).

  • comments table: Contains the comment data and a foreign key post_id which references the posts table.


Defining the Models

Next, we'll define the Post and Comment structs in CQL.

Post Model

struct Post
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :posts

  property id : Int64?
  property title : String
  property body : String
  property published_at : Time?

  # Initializing a new post with title, body, and optional published_at
  def initialize(@title : String, @body : String, @published_at : Time? = Time.utc)
  end
end

Comment Model

struct Comment
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :comments

  property id : Int64?
  property body : String
  property post_id : Int64?

  # Initializing a comment with a body. Post can be associated later.
  def initialize(@body : String, @post_id : Int64? = nil)
  end

  # Association: Each Comment belongs to one Post
  belongs_to :post, Post, foreign_key: :post_id
end

In the Comment model, we specify the belongs_to :post, Post, foreign_key: :post_id association. This links each comment to its parent post. The Comment model must have a post_id attribute (matching the foreign_key option) that stores the id of the associated Post.


Creating and Querying Records

Now that we have defined the Post and Comment models with a belongs_to relationship, let's see how to create and query records in CQL.

Creating Records

Option 1: Create Parent, then Child

# 1. Create and save the parent Post first
post = Post.new(title: "My First Blog Post", body: "This is the body of the post.")
post.save!

# 2. Create the Comment and associate it
#    a) By setting the foreign key directly:
comment1 = Comment.new(body: "Great post via direct FK!")
comment1.post_id = post.id # Assign the foreign key
comment1.save!
puts "Comment 1 associated with Post ID: #{comment1.post_id}, Post title: #{comment1.post.try(&.title)}"

#    b) By using the association setter:
comment2 = Comment.new(body: "Awesome article via association setter!")
comment2.post = post # Assign the Post instance to the association
comment2.save!
puts "Comment 2 associated with Post ID: #{comment2.post_id}, Post title: #{comment2.post.try(&.title)}"

Option 2: Create Child and Associated Parent Simultaneously (if needed) If you have a Comment instance and want to create its Post at the same time (less common for belongs_to primary creation flow but possible via association methods):

new_comment = Comment.new(body: "A comment for a brand new post.")
# This creates a new Post, saves it, and associates it with new_comment
created_post = new_comment.create_post(title: "Post Created Via Comment", body: "Content for post created via comment.")
new_comment.save! # Save the comment itself which now has the post_id populated

puts "New comment ID: #{new_comment.id}, associated Post ID: #{new_comment.post_id}"
puts "Title of post created via comment: #{created_post.title}"
puts "Comment's post title: #{new_comment.post.try(&.title)}"

Note: create_association (like create_post) will create and save the associated object (Post) and set the foreign key on the current object (new_comment). The current object itself (new_comment) still needs to be saved if it's new.

Option 3: Build Associated Parent (without saving parent yet)

yet_another_comment = Comment.new(body: "Comment with a built post.")
# This builds a new Post instance but does not save it to the DB yet.
# The foreign key on `yet_another_comment` is not set by `build_post`.
built_post = yet_another_comment.build_post(title: "Built, Not Saved Post", body: "Body of built post.")

# You would then save `built_post` and then `yet_another_comment` after setting association.
# built_post.save!
# yet_another_comment.post = built_post
# yet_another_comment.save!
puts "Built post title: #{built_post.title} (not yet saved)"

Querying the Associated Post from a Comment

Once we have a comment, we can retrieve the associated post using the belongs_to association.

crystalCopy code# Fetch the comment
comment = Comment.find(1)

# Fetch the associated post
post = comment.post

puts post.title  # Outputs: "My First Blog Post"

In this example, comment.post will fetch the Post associated with that Comment.


Summary

In this guide, we've covered the basics of the belongs_to relationship in CQL. We:

  • Defined the Post and Comment tables in the schema.

  • Created the corresponding models, specifying the belongs_to relationship in the Comment model.

  • Showed how to create and query records using the belongs_to association.

Next Steps

In the next guides, we'll build on this ERD and introduce other types of relationships like has_one, has_many, and many_to_many. Stay tuned for the next part where we'll cover the has_many relationship!

Feel free to play around with this setup and extend the models or experiment with more queries to familiarize yourself with CQL's Active Record capabilities.

FAQs

This page answers common questions about Crystal Query Language (CQL) and its Active Record implementation.


General CQL Questions

Q: What is CQL (Crystal Query Language)?

A: CQL is an Object-Relational Mapping (ORM) library for the Crystal programming language. It provides an abstraction over SQL databases, enabling developers to define and interact with relational data using Crystal objects and a type-safe query builder, rather than writing raw SQL queries directly for most operations.

Q: What are the key features of CQL?

A: Key features typically include:

  • Type-safe query building leveraging Crystal's static type system.

  • A macro-powered DSL for defining models and their mappings.

  • Support for multiple database adapters (e.g., PostgreSQL, MySQL, SQLite).

  • Migrations system for managing database schema changes.

  • An Active Record pattern implementation for model interaction (and flexibility for other patterns).

Q: How do I install CQL?

A: You can install CQL by adding it to your project's shard.yml file:

dependencies:
  cql:
    github: azutoolkit/cql # Or the appropriate source for your CQL version
    version: "~> x.y.z" # Specify the version you are using

Then, run shards install in your terminal to download and install the dependency. For more setup details, see the Active Record Setup Guide.

Q: Which databases does CQL support?

A: CQL is designed to support major relational databases through Crystal DB drivers. Commonly supported databases include PostgreSQL, MySQL, and SQLite. Check the specific CQL version documentation for the most up-to-date list and any driver requirements.


Active Record Specific Questions

Q: What is the Active Record pattern in CQL?

A: The Active Record pattern maps database tables to Crystal structs (models). Each instance of a model corresponds to a row in the table. This pattern provides methods directly on the model and its instances for database interaction (CRUD operations, queries, etc.).

  • For a conceptual overview, see Active Record Pattern Concept.

  • For a detailed guide on using it with CQL, see Active Record with CQL Guide.

Q: How do I define an Active Record model in CQL?

A: You define a model by creating a Crystal struct, including CQL::ActiveRecord::Model(PrimaryKeyType), and using the db_context macro to link it to a database table and context.

  • Learn more in the Defining Models Guide.

Q: How do I perform basic Create, Read, Update, and Delete (CRUD) operations?

A: CQL Active Record provides intuitive methods like new/save, create!, find?, find_by!, update!, and delete! directly on your models and their instances.

  • For comprehensive examples, see the CRUD Operations Guide.

Q: How can I build more complex queries?

A: CQL offers a chainable query interface. You can start with Model.query or methods like Model.where(...) and then chain further conditions like .order(), .limit(), .join(), etc.

  • Explore the Querying Guide for detailed information.

Q: How do I handle database relationships (e.g., has_many, belongs_to)?

A: CQL Active Record uses macros like has_many, belongs_to, has_one, and many_to_many to define associations between models.

  • See the Relations section in the main Active Record guide for links to detailed guides on each relationship type:

    • belongs_to

    • has_one

    • has_many

    • many_to_many

Q: How are validations handled in CQL Active Record?

A: You can define validations in your model to ensure data integrity (e.g., presence, length, format). Invalid records will not be saved, and errors can be inspected on the model instance.

  • Refer to the Validations Guide.

Q: What are callbacks and how do I use them?

A: Callbacks are methods that get triggered at specific points in a model's lifecycle (e.g., before_save, after_create). They allow you to run custom logic automatically.

  • Learn how to use them in the Callbacks Guide.

Q: What's the difference between save and save! (or create and create!)?

A: Methods ending with ! (bang methods) typically raise an exception if the operation fails (e.g., a validation error occurs or the record isn't found). Methods without ! usually return false or nil on failure, allowing you to handle errors programmatically without a begin/rescue block for common cases.

  • Example: user.save returns true or false. user.save! returns true or raises an exception (e.g., CQL::Errors::RecordInvalid).

Q: How does CQL handle database migrations for Active Record models?

A: CQL includes a migration system where you define schema changes in Crystal classes (e.g., creating tables, adding columns). These migrations can be run to update your database schema in a version-controlled manner.

  • For details, see the Database Migrations Guide for Active Record.

  • For a general overview of migrations in CQL, see Core Concepts: Migrations.

Q: Does CQL Active Record support database transactions?

A: Yes, database transactions are crucial for ensuring data consistency, especially when multiple database operations need to succeed or fail together. CQL typically provides a way to manage transactions.

  • See the Transaction Management Guide for more details (Note: Ensure this guide specifically covers Active Record transaction patterns if applicable, or link to a more general CQL transaction guide if that's more appropriate).


Troubleshooting and Further Information

Q: Where can I find more detailed documentation on Active Record features?

A: The primary resource is the Active Record with CQL main guide, which links to all specialized guides covering different aspects of the Active Record implementation.

Q: I think I found a bug or want to suggest a feature. How do I proceed?

A: Typically, you would report issues or suggest features via the GitHub repository for CQL (e.g., azutoolkit/cql or the specific fork/version you are using). Look for an "Issues" tab to see if similar issues exist or to create a new one. Contributions via Pull Requests are also often welcome after discussing the proposed changes.

Q: Is there a community or forum for CQL users?

A: Check the official CQL repository or related Crystal community channels (like the Crystal forum, Discord, or Gitter) for discussions, help, and community support related to CQL.


If your question isn't answered here, please check the other documentation sections or consider reaching out to the CQL community.

Active Record with CQL

An overview of CQL's Active Record capabilities for defining models, interacting with your database, managing data integrity, and more.

This guide provides a comprehensive overview of Crystal Query Language (CQL)'s Active Record implementation. Active Record is a design pattern that connects database tables to classes (or structs in Crystal), allowing you to interact with your data through objects and methods rather than raw SQL queries.

CQL's Active Record module offers a powerful and intuitive way to manage your database records, inspired by established ORMs while leveraging Crystal's type safety and performance.


Core Concepts & Guides

This central README provides a high-level introduction. For in-depth information on specific aspects of CQL Active Record, please refer to the following guides:

  • Setup and Prerequisites: Initial configuration for using CQL and Active Record. (Covered below)

  • Defining Models: Learn how to define your Active Record models, map them to database tables, specify primary keys, and work with attributes.

  • CRUD Operations: Detailed guide on creating, reading, updating, and deleting records using Active Record methods.

  • Querying: Explore the powerful query interface, including direct finders, chainable queries, aggregations, and scopes.

  • Transactions: Ensure data integrity by using database transactions for multi-step operations.

  • Persistence Details: Understand how to check if a record is persisted and how to reload its data from the database.

  • Validations: Ensure data integrity by defining and using model validations.

  • Callbacks: Hook into the lifecycle of your models to trigger logic at specific events (e.g., before save, after create).

  • Relations: Define and use associations between models:

    • belongs_to

    • has_one

    • has_many

    • many_to_many (covers has_and_belongs_to_many)

  • Database Migrations: Manage your database schema changes over time.

  • Scopes: Define reusable query constraints for cleaner and more readable code.

  • Pagination: Easily paginate query results.


Prerequisites and Setup

Before getting started, ensure you have the following:

  • Crystal language installed (latest stable version recommended).

  • A supported relational database (e.g., PostgreSQL, MySQL) set up and accessible.

  • CQL added to your Crystal project.

Adding CQL to Your Project

Include CQL in your project's shard.yml:

dependencies:
  cql:
    github: azutoolkit/cql # Or the appropriate source for your CQL version
    version: "~> x.y.z" # Specify the version you are using

Then, run shards install to download and install the dependency.

Database Connection Setup

You need to configure CQL to connect to your database. This is typically done by setting a database URL and opening a connection. You might also define a database context for your application.

require "cql"

# Example: Define your database connection URL (replace with your actual credentials)
# For PostgreSQL:
ENV_DB_URL = ENV["DATABASE_URL"]? || "postgres://username:password@localhost:5432/myapp_development"

# Define a database context. This is often a class or module that your models will reference.
# The name `AcmeDB` is used as a placeholder in these guides.
module AcmeDB
  # Establishes and memoizes the database connection.
  def self.db
    @@db ||= DB.open(ENV_DB_URL)
  end

  # Optional: A method to close the connection if needed during shutdown or testing.
  def self.close_db
    @@db.try(&.close)
    @@db = nil
  end
end

# Ensure your models can reference this context, e.g.:
struct User
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :users
  # ...
end

Note: The exact mechanism for defining your database context (AcmeDB in the example) and making it accessible to your models should align with CQL's specific API and your application structure. Refer to CQL's core documentation for advanced database connection management, pooling, and context configuration.


Quick Overview of Key Features

Defining Models

Models are Crystal structs including CQL::ActiveRecord::Model(PkType) and use db_context to link to a table.

See the full Defining Models Guide for details on attributes, primary keys, and more.

CRUD Operations

CQL provides intuitive methods for creating, reading, updating, and deleting records (e.g., save, create!, find?, find_by!, update!, delete!).

Explore the CRUD Operations Guide for comprehensive examples.

Querying

Fetch records using direct finders or build complex queries with a chainable interface (.where, .order, .limit, etc.).

Dive into the Querying Guide for all query-building capabilities.

Transactions

Maintain data integrity with ACID-compliant database transactions. CQL provides both model-level transaction support and a service objects pattern for complex operations.

# Basic transaction usage
BankAccount.transaction do |tx|
  account = BankAccount.find(1)
  account.balance -= 100
  account.save!

  Transaction.create!(
    amount: 100,
    transaction_type: "withdrawal",
    from_account_id: account.id,
    created_at: Time.utc
  )

  # All operations succeed or fail together
end

Learn more in the Transactions Guide for maintaining data integrity across multiple operations.

Validations

Ensure data integrity with built-in or custom validation rules triggered before saving records.

Learn more in the Validations Guide.

Callbacks

Execute custom logic at different points in a model's lifecycle (e.g., before_save, after_create).

Consult the Callbacks Guide for usage details.

Relations

Define associations like belongs_to, has_many, has_one, and many_to_many to manage relationships between models.

  • belongs_to

  • has_one

  • has_many

  • many_to_many

Migrations

Manage database schema changes systematically using Crystal-based migration files.

See the Database Migrations Guide for how to write and run migrations.

Scopes

Create reusable query shortcuts to keep your code clean and expressive.

Read the Scopes Guide for defining and using scopes.

Optimistic Locking

Optimistic locking is a concurrency control strategy that allows multiple transactions to proceed without blocking each other, while still preventing conflicts from concurrent updates. CQL provides built-in support for optimistic locking through a version column mechanism.

How It Works

  1. A version column (e.g., an integer) is added to your database table.

  2. When a record is read, its current version number is also fetched.

  3. When an attempt is made to update the record, the ORM includes a condition in the UPDATE statement to check if the version number in the database is still the same as when it was fetched.

  4. Simultaneously, the UPDATE statement also increments this version number.

  5. If the version number in the database has changed (meaning another process updated the record in the meantime), the WHERE condition (e.g., WHERE id = ? AND version = ?) will not match any rows. The update will affect zero rows.

  6. The ORM detects that no rows were affected and raises an OptimisticLockError, signaling that a concurrent update occurred.

  7. The application must then handle this error, typically by reloading the record (to get the new version and latest data) and retrying the update.

When to Use Optimistic Locking

Optimistic locking is particularly well-suited for scenarios where:

  • Conflicts are Rare: You expect data conflicts to be infrequent. If conflicts are very common, the overhead of retrying transactions might outweigh the benefits.

  • High Concurrency is Required: The application needs to support many users or processes accessing the same data concurrently, and you want to avoid the performance bottlenecks associated with pessimistic locking (which locks records for the duration of a transaction).

  • Read-Heavy Workloads: Read operations are significantly more frequent than write operations. Optimistic locking doesn't impose any overhead on reads.

  • User-Facing Applications: In applications where a user might have data форм open for a while before submitting changes, optimistic locking can prevent their changes from unknowingly overwriting updates made by others in the interim.

  • Disconnected or Stateless Environments: Holding database locks across requests in a stateless web application or a distributed system can be complex or impractical. Optimistic locking provides a way to manage concurrency without long-lived locks.

  • Cost of Retry is Low: The business logic can tolerate and gracefully handle retries. For example, if updating a product description fails due to a conflict, reloading and trying again is often acceptable.

It's generally not recommended if:

  • Conflicts are very frequent, as this will lead to many retries and potentially poor performance.

  • The cost of a failed transaction retry is very high (e.g., involving complex external API calls that are not idempotent).

Setting Up Optimistic Locking

1. Add a Version Column to Your Table

When defining your table schema, add a version column using the lock_version method:

table :users do
  primary :id, Int64
  varchar :name, String
  varchar :email, String
  lock_version :version # Adds an integer column named 'version' with default value 1
end

The lock_version method accepts the following parameters:

  • name (Symbol): The name of the column (default: :version). This column should typically be an Int32 or Int64.

  • as_name (String?): An optional alias for the column in queries (rarely needed for version columns).

  • null (Bool): Whether the column can be null (default: false). It's crucial that version columns are not nullable.

  • default (DB::Any): The default value for new records (default: 1).

  • index (Bool): Whether to create an index on this column (default: false). Indexing might be beneficial if you query by version, but it's not strictly necessary for the locking mechanism itself.

2. Use Optimistic Locking with Raw Queries

When using raw queries, you can use the with_optimistic_lock method on the Update object:

# Assuming 'schema' is your CQL::Schema instance
# And we have a record with id=1 and its current version is 5
begin
  update_result = CQL::Update.new(schema)
    .table(:users)
    .set(name: "John Doe", age: 31) # age is an example, ensure your table has it
    .where(id: 1)
    .with_optimistic_lock(version: 5) # Checks for version 5, will set to 6
    .commit

  if update_result.rows_affected == 1
    puts "Update successful!"
  end
rescue CQL::OptimisticLockError
  puts "Conflict detected! Record was updated by another process."
  # Implement retry logic: reload data and try update again
end

If another process has already updated the record (version is no longer 5), a CQL::OptimisticLockError will be raised by the .commit call (or specifically, by the underlying mechanism in CQL::Update when with_optimistic_lock is used and rows_affected is 0).

3. Use Optimistic Locking with Active Record

For Active Record models, include the CQL::ActiveRecord::OptimisticLocking module and define which column to use with the optimistic_locking macro:

class User < CQL::ActiveRecord::Base # Assuming Base is your base AR class
  include CQL::ActiveRecord::OptimisticLocking

  # db_context points to your schema and table
  db_context MY_SCHEMA, :users # Replace MY_SCHEMA with your actual schema variable

  # Define properties matching your table columns
  property id : Int64?
  property name : String?
  property email : String?
  property version : Int32? # This will hold the lock version

  # Configure optimistic locking
  optimistic_locking version_column: :version
end

This will automatically handle version checking and incrementing when updating records:

user = User.find!(1)
user.name = "Johnathan Doe"
begin
  user.update! # Uses optimistic locking automatically
  puts "User updated successfully. New version: #{user.version}"
rescue CQL::OptimisticLockError
  puts "Failed to update user: Optimistic lock conflict."
  # Handle the conflict, e.g., reload and retry
  user.reload!
  puts "Retrying update after reloading. Current version: #{user.version}"
  user.name = "Johnathan Doe" # Re-apply changes if necessary
  user.update! # Retry the update
  puts "User updated successfully on retry. New version: #{user.version}"
end

If a concurrent update has occurred, user.update! will raise a CQL::OptimisticLockError.

Handling Concurrent Updates

When a CQL::OptimisticLockError is caught, your application needs to decide how to proceed. Common strategies include:

  1. Reload and Retry:

    • Reload the record from the database to get the latest version and data.

    • Re-apply the intended changes to the newly reloaded record.

    • Attempt the update again.

    • You might want to limit the number of retries to avoid infinite loops in high-contention scenarios.

    user = User.find!(1)
    max_retries = 3
    attempt = 0
    
    loop do
      attempt += 1
      user.name = "John Doe Updated" # Apply desired changes
      begin
        user.update!
        puts "Update successful!"
        break # Exit loop on success
      rescue CQL::OptimisticLockError
        if attempt >= max_retries
          puts "Max retries reached. Could not update record."
          # Potentially notify user or log a more persistent error
          break
        end
        puts "Optimistic lock conflict. Reloading and retrying (attempt #{attempt}/#{max_retries})..."
        user.reload! # Reloads the record's attributes, including the version
      end
    end
  2. Inform the User:

    • If changes are being made through a UI, inform the user that the data has changed since they last viewed it.

    • Show them the updated data and allow them to decide whether to overwrite, merge their changes, or discard their changes.

  3. Merge Changes (Advanced):

    • If possible and logical, attempt to merge the conflicting changes. This is highly application-specific and can be complex. For example, if one user changed the product description and another changed the price, these changes might be mergeable.

Best Practices

  1. Use Transactions (Where Appropriate): While optimistic locking handles concurrent updates to a single record, if your operation involves multiple database changes that must be atomic, ensure these are wrapped in a database transaction. Optimistic locking complements, rather than replaces, the need for transactions for atomicity of complex operations.

  2. Handle Conflicts Gracefully: Provide a clear and understandable user experience when conflicts occur. Avoid cryptic error messages. Explain that the data was changed by someone else and suggest next steps.

  3. Selective Usage: Apply optimistic locking to entities where concurrent modification is a realistic concern and data integrity is critical. Not every table or resource needs it. Overusing it can add unnecessary complexity.

  4. Always Reload on Conflict: Before retrying an update after a conflict, always reload the record. This ensures you have the latest version number and the most current state of other attributes, preventing further issues or lost updates.

  5. Combine with Other Techniques (If Needed): For highly contended resources where optimistic lock failures become too frequent, you might need to explore other strategies for those specific parts of your application, such as more granular locking, CQRS patterns, or even considering pessimistic locking as a last resort if the cost of retries is too high.

  6. Choose the Right Version Column Type: Typically, an INTEGER or BIGINT (Int32 or Int64 in Crystal) is suitable. Ensure it won't realistically overflow during the lifetime of a record.

  7. Atomic Increment by Database: Rely on the database's atomicity for the UPDATE ... SET version = version + 1 WHERE version = old_version pattern. CQL's implementation handles this.

  8. Thorough Testing: Write specific tests for conflict scenarios to ensure your conflict resolution logic works as expected. Simulate concurrent updates in your tests.

  9. Performance Monitoring: Monitor the frequency of OptimisticLockErrors. A consistently high rate might indicate that the "optimistic" assumption (conflicts are rare) is not holding for that particular resource, and a different concurrency strategy might be more efficient.

Database-Specific Considerations

Optimistic locking, as implemented with a version column incremented by the application (via the ORM), is a widely portable pattern that works consistently across most relational databases. The core mechanism relies on standard SQL features (UPDATE with a WHERE clause checking the version and SET to increment it).

Here's how it generally applies to common databases used with Crystal:

PostgreSQL

  • Behavior: Works seamlessly. PostgreSQL's robust MVCC (Multi-Version Concurrency Control) architecture is well-suited for optimistic locking.

  • Version Column Type: INTEGER or BIGINT are excellent choices.

  • Transactions: PostgreSQL has strong ACID-compliant transactions. Use them to group related operations where atomicity is required, in conjunction with optimistic locking for individual record updates.

  • No Special Configuration: No special database-level configuration is typically needed for CQL's optimistic locking to function correctly.

MySQL

  • Behavior: Works well, especially with the InnoDB storage engine (which is the default for modern MySQL versions and provides row-level locking and transactions).

  • Version Column Type: INT or BIGINT are appropriate.

  • Transactions: InnoDB supports ACID transactions. As with PostgreSQL, use transactions for atomicity of larger operations.

  • Isolation Levels: While optimistic locking itself relies on the version check, ensure your transaction isolation level (e.g., REPEATABLE READ or READ COMMITTED for InnoDB) meets your application's general consistency requirements. Optimistic locking can help reduce the need for stricter isolation levels for the sole purpose of preventing lost updates.

SQLite3

  • Behavior: Optimistic locking is fully functional and effective with SQLite.

  • Version Column Type: INTEGER is standard.

  • Concurrency Model: SQLite, by default, serializes writes at the database level (multiple connections can read concurrently, but writes are typically queued). However, if your application has multiple threads or processes accessing the same connection without proper external synchronization, or if multiple independent processes access the same database file, optimistic locking is still crucial to prevent lost updates.

  • WAL Mode: Using Write-Ahead Logging (WAL) mode in SQLite can improve concurrency for readers and a single writer. Optimistic locking remains a valuable pattern even with WAL mode to manage conflicts if multiple writers could conceptually attempt updates based on stale data.

  • Simplicity: For simpler applications or embedded use cases where SQLite shines, optimistic locking provides a straightforward way to handle potential concurrency issues without complex locking mechanisms.

In summary: CQL's optimistic locking implementation is designed to be database-agnostic by relying on standard SQL patterns. The primary responsibility of the database is to atomically execute the conditional UPDATE statement. The choice of database mainly influences broader aspects like overall transaction management, concurrency capabilities, and performance characteristics, but the optimistic locking logic itself remains consistent.

Limitations

  • Increased Retries: Not suitable for high-contention resources where conflicts would occur very frequently, as this leads to many transaction rollbacks and retries, potentially degrading performance.

  • Application-Level Handling: Requires application logic to catch the OptimisticLockError and implement a retry or conflict resolution strategy. It's not transparent like some pessimistic locking mechanisms.

  • Slight Write Overhead: Each update involves checking and incrementing the version column, which is a minimal but present overhead compared to an update without version checking.

  • "Last Commit Wins" (Effectively): While it prevents lost updates, the record will reflect the state of the last successful commit. If two users make conflicting changes, the first one to commit "wins" for that version, and the second must handle the conflict.

Handling Migrations

Migrations in CQL provide a structured way to evolve your database schema over time. They allow you to create, modify, and drop tables and columns in a versioned, reversible manner using Crystal code.


What is a Migration?

A migration is a Crystal class that defines changes to your database schema. Each migration has a unique version number and two methods:

  • up: Applies the migration (e.g., creates or alters tables).

  • down: Reverts the migration (e.g., drops or reverts changes).


Creating a Migration

Create a new migration by subclassing CQL::Migration and setting a unique version:

class CreateUsers < CQL::Migration
  self.version = 1_i64

  def up
    schema.create :users do
      primary_key :id, :serial
      column :name, :string
      column :email, :string
      column :active, :bool, default: false
      column :created_at, :timestamp
      column :updated_at, :timestamp
    end
  end

  def down
    schema.drop :users
  end
end
  • self.version must be unique and increasing (often a timestamp or integer).

  • The up method defines the schema changes to apply.

  • The down method defines how to revert those changes.


Running Migrations

You can run migrations using your preferred migration runner or a custom script. A typical workflow:

  1. Place migration files in a db/migrations/ directory.

  2. Load and apply migrations in order:

require "cql"

# Load all migration files
dir = "./db/migrations"
Dir.glob("#{dir}/*.cr").each { |file| require file }

# Run all migrations (pseudo-code, depends on your runner)
CQL::Migrator.new(AcmeDB).migrate!

Consult your project's migration runner or CQL's documentation for details on migration management.


Modifying Tables

You can alter existing tables in a migration:

class AddAgeToUsers < CQL::Migration
  self.version = 2_i64

  def up
    schema.alter :users do
      add_column :age, :int32
    end
  end

  def down
    schema.alter :users do
      drop_column :age
    end
  end
end

Best Practices

  • Use a unique, increasing version for each migration.

  • Write reversible migrations (always provide a down method).

  • Keep migrations in version control.

  • Test migrations on a development database before running in production.

  • Use descriptive class names (e.g., AddAgeToUsers).


Example: Multiple Migrations

class CreatePosts < CQL::Migration
  self.version = 3_i64

  def up
    schema.create :posts do
      primary_key :id, :serial
      column :user_id, :int32
      column :title, :string
      column :body, :string
      column :created_at, :timestamp
    end
  end

  def down
    schema.drop :posts
  end
end

class AddIndexToUsersEmail < CQL::Migration
  self.version = 4_i64

  def up
    schema.alter :users do
      add_index :email, unique: true
    end
  end

  def down
    schema.alter :users do
      drop_index :email
    end
  end
end

For more on defining models and querying, see the other guides in this directory.

Validations

CQL's Active Record integration provides a simple yet effective way to ensure data integrity through model-level validations. By defining validation logic within your models, you can prevent invalid data from being persisted to the database.

The CQL::ActiveRecord::Validations module is automatically included when you use CQL::ActiveRecord::Model(Pk).


Defining Validations

Validations are typically implemented by overriding the validate instance method in your model. This method is called automatically before saving a record (create or update).

Inside the validate method, you should check the model's attributes. If an attribute is invalid, you add a message to the errors collection for that attribute.

Model Example:

struct User
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :users

  property id : Int64?
  property name : String
  property email : String
  property age : Int32?

  def initialize(@name : String, @email : String, @age : Int32? = nil)
  end

  # Override the validate method to add custom validation logic
  def validate
    super # It's good practice to call super in case parent classes have validations

    if name.empty?
      errors.add(:name, "cannot be blank")
    end

    if email.empty?
      errors.add(:email, "cannot be blank")
    elsif !email.includes?('@')
      errors.add(:email, "is not a valid email format")
    end

    if age_val = age
      if age_val < 18
        errors.add(:age, "must be 18 or older")
      end
    end
  end
end

Checking Validity

You can explicitly check if a model instance is valid using the valid? and validate! methods.

valid?

The valid? method runs the validate method and returns true if the errors collection is empty, and false otherwise. It does not raise an exception.

user = User.new(name: "", email: "test")
user.valid? # => false
puts user.errors.full_messages.join(", ") # => "Name cannot be blank, Email is not a valid email format"

valid_user = User.new(name: "Jane Doe", email: "jane@example.com", age: 30)
valid_user.valid? # => true
puts valid_user.errors.empty? # => true

validate!

The validate! method also runs the validate method. If the record is invalid, it raises a CQL::Errors::RecordInvalid exception containing the validation errors. If the record is valid, it returns true.

user = User.new(name: "", email: "test")
begin
  user.validate!
rescue ex : CQL::Errors::RecordInvalid
  puts ex.message # Displays the summary of errors
  puts ex.record.errors.full_messages.join(", ") # => "Name cannot be blank, Email is not a valid email format"
end

valid_user = User.new(name: "John Doe", email: "john@example.com", age: 25)
valid_user.validate! # => true (no exception raised)

The errors Object

The errors object is an instance of CQL::Errors::Collection. It provides methods to add and inspect error messages.

  • errors.add(attribute : Symbol, message : String): Adds an error message for the specified attribute.

  • errors.empty?: Returns true if there are no errors.

  • errors.clear: Removes all error messages.

  • errors.on(attribute : Symbol): Returns an array of error messages for a specific attribute, or nil if none.

  • errors.full_messages: Returns an array of user-friendly error messages, typically in the format "Attribute_name message" (e.g., "Name cannot be blank").

  • errors.to_h: Returns a hash where keys are attribute names (Symbols) and values are arrays of error messages for that attribute.

user = User.new(name: "", email: "bademail", age: 10)
user.valid? # Run validations to populate errors

puts user.errors.on(:name)        # => ["cannot be blank"]
puts user.errors.on(:email)       # => ["is not a valid email format"]
puts user.errors.on(:age)         # => ["must be 18 or older"]
puts user.errors.on(:non_existent) # => nil

puts user.errors.full_messages
# [
#   "Name cannot be blank",
#   "Email is not a valid email format",
#   "Age must be 18 or older"
# ]

pp user.errors.to_h
# {
#   name: ["cannot be blank"],
#   email: ["is not a valid email format"],
#   age: ["must be 18 or older"]
# }

Validations and Persistence Methods

Validation is automatically integrated with the persistence methods:

  • save / save!:

    • save runs validations. If they fail (valid? returns false), save returns false and the record is not persisted.

    • save! also runs validations. If they fail, it raises CQL::Errors::RecordInvalid.

  • create / create!:

    • create(attributes) is like new(attributes).save. It returns the instance (which may be invalid and not persisted) or false if using a before_create callback that halts.

    • create!(attributes) is like new(attributes).save!. It raises CQL::Errors::RecordInvalid if validations fail.

  • update(attributes) / update!(attributes):

    • update assigns attributes and then calls save. Returns true or false.

    • update! assigns attributes and then calls save!. Raises CQL::Errors::RecordInvalid on failure.

Example with save:

invalid_user = User.new(name: "", email: "test")
if invalid_user.save
  puts "User saved successfully!" # This won't be printed
else
  puts "Failed to save user."
  puts invalid_user.errors.full_messages.join(", ")
  # => Failed to save user.
  # => Name cannot be blank, Email is not a valid email format
end

valid_user = User.new(name: "Alice", email: "alice@example.com", age: 30)
if valid_user.save
  puts "User '#{valid_user.name}' saved successfully! ID: #{valid_user.id.not_nil!}"
else
  puts "Failed to save valid user." # This won't be printed
end

Example with create!:

begin
  User.create!(name: "", email: "invalid")
rescue ex : CQL::Errors::RecordInvalid
  puts "Caught error on create!: #{ex.message}"
end

new_user = User.create!(name: "Bob", email: "bob@example.com", age: 40)
puts "User '#{new_user.name}' created successfully! ID: #{new_user.id.not_nil!}"

Custom Validation Helpers (Optional)

For more complex or reusable validation logic, you can define private helper methods within your model.

struct Product
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :products

  property id : Int64?
  property name : String
  property price : Float64
  property sku : String

  def initialize(@name : String, @price : Float64, @sku : String)
  end

  def validate
    super
    validate_name
    validate_price
    validate_sku_format
  end

  private def validate_name
    errors.add(:name, "cannot be empty") if name.empty?
  end

  private def validate_price
    errors.add(:price, "must be positive") if price <= 0
  end

  # Example: SKU must be in format ABC-12345
  private def validate_sku_format
    unless sku.matches?(/^[A-Z]{3}-\d{5}$/)
      errors.add(:sku, "format is invalid (expected ABC-12345)")
    end
  end
end

# Example usage:
product = Product.new(name: "Test Product", price: -10.0, sku: "XYZ-123")
product.valid? # => false
pp product.errors.to_h
# {
#   price: ["must be positive"],
#   sku: ["format is invalid (expected ABC-12345)"]
# }

By using these validation features, you can maintain data consistency and provide clear feedback about data entry issues. Remember that CQL::ActiveRecord::Validations provides a basic framework. For very complex validation scenarios or integrating with external validation libraries, you might need to extend this functionality further.

Callbacks

CQL's Active Record provides lifecycle callbacks that allow you to trigger logic at various points during an object's life. This is useful for tasks like data normalization before validation, encrypting passwords before saving, or sending notifications after an action.

The CQL::ActiveRecord::Callbacks module is automatically included when you use CQL::ActiveRecord::Model(Pk).


Available Callbacks

Callbacks are methods defined in your model that are registered to be called at specific moments. CQL supports the following callbacks:

Validation Callbacks:

  • before_validation(method_name): Called before validate is run.

  • after_validation(method_name): Called after validate completes.

Save Callbacks (run for both create and update):

  • before_save(method_name): Called before the record is saved to the database.

  • after_save(method_name): Called after the record is saved to the database.

Create Callbacks (run only when a new record is saved):

  • before_create(method_name): Called before a new record is inserted into the database.

  • after_create(method_name): Called after a new record is inserted into the database.

Update Callbacks (run only when an existing record is saved):

  • before_update(method_name): Called before an existing record is updated in the database.

  • after_update(method_name): Called after an existing record is updated in the database.

Destroy Callbacks:

  • before_destroy(method_name): Called before a record is deleted from the database.

  • after_destroy(method_name): Called after a record is deleted from the database.


Registering Callbacks

You register a callback by calling its macro with the name of the method (as a Symbol) to be executed.

In this example:

  • normalize_email will run before validations.

  • set_status_if_nil will run before any save operation (create or update).

  • send_welcome_email will run only after a new user is created.

  • record_login_time will run only before an existing user is updated.

  • log_deletion will run after a user is destroyed.


Halting Execution

If a before_validation, before_save, before_create, before_update, or before_destroy callback method returns false (explicitly false, not nil or other falsy values), the callback chain is halted. This means:

  • Subsequent callbacks of the same type (e.g., other before_save methods) will not be executed.

  • The main action (validation, save, create, update, or destroy) will be canceled.

    • For save, create, update, it will return false.

    • For save!, create!, update!, it will not raise RecordInvalid due to validation errors (if any ran), but simply won't proceed with persistence.

    • For destroy, it will return false, and the record will not be deleted.

Example of halting:

after_* callbacks do not have the power to halt the chain, as the primary action has already completed.


Order of Callbacks

When multiple callbacks are registered for the same event, they are executed in the order they were defined in the model.

CQL aims to follow a similar callback order to other popular ORMs like Rails Active Record during the save process:

  1. before_validation

  2. Validations are run (validate method)

  3. after_validation

  4. before_save

  5. If new record: before_create

  6. If existing record: before_update

  7. Database operation (INSERT or UPDATE)

  8. If new record: after_create

  9. If existing record: after_update

  10. after_save

For destroy:

  1. before_destroy

  2. Database operation (DELETE)

  3. after_destroy


Use Cases

  • Data Manipulation: Normalize data (e.g., downcasing emails), set default values, generate tokens.

  • Lifecycle Management: Update related objects, log changes, manage state transitions.

  • Notifications: Send emails or push notifications after certain events (e.g., after_create).

  • Conditional Logic: A callback method can contain logic to decide if it should perform an action, or even halt the entire operation.

Callbacks are a powerful tool for adding behavior to your models without cluttering your controller or service logic. However, use them judiciously, as complex callback chains can sometimes make debugging harder.

struct User
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :users

  property id : Int64?
  property name : String
  property email : String
  property status : String?
  property last_logged_in_at : Time?

  def initialize(@name : String, @email : String)
    @status = "pending"
  end

  # Registering callbacks
  before_validation :normalize_email
  before_save :set_status_if_nil
  after_create :send_welcome_email
  before_update :record_login_time
  after_destroy :log_deletion

  private def normalize_email
    self.email = email.downcase.strip
    true # Important: before_ callbacks must return true or a truthy value to continue
  end

  private def set_status_if_nil
    self.status = "active" if status.nil?
    true
  end

  private def send_welcome_email
    puts "Sending welcome email to #{email}..."
    # In a real app, you'd use an email library here
    true
  end

  private def record_login_time
    # Hypothetical: this callback might be triggered indirectly
    # if an update operation is intended to signify a login.
    # More commonly, you'd set this explicitly.
    self.last_logged_in_at = Time.utc
    true
  end

  private def log_deletion
    puts "User #{name} (ID: #{id.inspect}) deleted."
    true # after_ callbacks don't halt the chain, but consistency is good
  end
end
struct Article
  include CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :articles

  property id : Int64?
  property title : String
  property published : Bool

  def initialize(@title : String, @published : Bool = false)
  end

  before_save :check_if_can_be_published

  private def check_if_can_be_published
    if published && title.empty?
      errors.add(:title, "cannot be blank if published")
      return false # Halt the save
    end
    true
  end
end

# Attempt to save an article that will be halted
article1 = Article.new(title: "", published: true)
if article1.save
  puts "Article 1 saved!" # This won't run
else
  puts "Article 1 not saved. Errors: #{article1.errors.full_messages.join(", ")}"
  # Output: Article 1 not saved. Errors: Title cannot be blank if published
end

# Attempt to save a valid article
article2 = Article.new(title: "My Great Article", published: true)
if article2.save
  puts "Article 2 saved! ID: #{article2.id.inspect}"
else
  puts "Article 2 not saved."
end

Relations

This section provides comprehensive guides for defining and working with associations between models in CQL Active Record. Relationships are a core part of modeling real-world data and enable you to express connections such as ownership, membership, and linkage between records.

What You'll Find

  • Overview of Association Types: Learn the differences between one-to-one, one-to-many, and many-to-many relationships.

  • Step-by-Step Guides: Each sub-guide covers how to set up, use, and query each type of association in CQL, with practical code examples.

  • Best Practices: Tips for managing associations, avoiding common pitfalls, and writing maintainable code.

Relationship Guides

  • Belongs To: How to set up and use belongs_to associations (e.g., a comment belongs to a post).

  • Has One: How to define and work with has_one relationships (e.g., a user has one profile).

  • Has Many: How to manage has_many associations (e.g., a post has many comments).

  • Many To Many: How to implement many-to-many relationships using join tables (e.g., posts and tags).

Each guide includes:

  • Schema and model setup

  • Creating and querying associations

  • Modifying and deleting related records

  • Eager loading and performance tips


Use these guides to master associations in your CQL-powered Crystal applications!

Many To Many

CQL Active Record: ManyToMany Relationship Guide

In this guide, we'll cover the ManyToMany relationship using CQL's Active Record syntax. This relationship is used when multiple records in one table can relate to multiple records in another table, facilitated by an intermediate join table.


What is a ManyToMany Relationship?

A ManyToMany relationship means that multiple records in one table can relate to multiple records in another table. For example:

  • A Post can have multiple Tags (e.g., "Tech", "News").

  • A Tag can belong to multiple Posts (e.g., both Post 1 and Post 2 can have the "Tech" tag).

Example Scenario: Posts and Tags

We'll use a scenario where:

  • A Post can have many Tags.

  • A Tag can belong to many Posts.

We will represent this many-to-many relationship using a join table called PostTags.


Defining the Schema

We'll define the posts, tags, and post_tags tables in the schema using CQL's DSL.

  • posts table: Stores post details such as title, body, and published_at.

  • tags table: Stores tag names.

  • post_tags table: A join table that connects posts and tags via their foreign keys post_id and tag_id.


Defining the Models

Let's define the Post, Tag, and PostTag models in CQL, establishing the ManyToMany relationship.

Post Model

  • The many_to_many :tags, Tag, join_through: :post_tags association in the Post model connects Post to Tag via the post_tags table.

Tag Model

  • Similarly, the Tag model uses many_to_many :posts, Post, join_through: :post_tags.

PostTag Model (Join Model)

  • The PostTag model is crucial. It belongs_to both Post and Tag.

  • The many_to_many macro uses this join model implicitly via the join_through: :post_tags option, which refers to the table name.


Working with ManyToMany Associations

When you access a many_to_many association (e.g., post.tags), you get a ManyCollection proxy that offers powerful methods to manage the relationship.

Creating and Associating Records

Accessing Associated Records

Removing Associations / Deleting Records

Removing an association (deletes the join table record, not the Tag itself):

Clearing all associations for a post (deletes all its PostTag records):

  • clear only removes the join records. The Tag records themselves are not deleted.

  • If cascade: true was set on the many_to_many association, the behavior of delete or clear with respect to the target records (Tag) might change, potentially deleting them. This needs careful checking of ManyCollection's cascade implementation.

Deleting a Post or a Tag will not automatically delete its associations from the post_tags table or the associated records on the other side, unless cascade deletes are configured at the database level or handled by before_destroy callbacks manually cleaning up join table records.


Eager Loading ManyToMany

To avoid N+1 queries with many-to-many associations, use includes:


Summary

In this guide, we explored the ManyToMany relationship in CQL:

  • Defined Post, Tag, and PostTag (join) models using CQL::ActiveRecord::Model(Pk).

  • Used the many_to_many :association, TargetModel, join_through: :join_table_symbol macro.

  • Showcased managing associations using ManyCollection methods like <<, create, delete, and clear.

  • Discussed accessing associated records and eager loading with includes.

This provides a robust way to handle many-to-many relationships in your Crystal applications using CQL.

Next Steps

This concludes our series of guides on relationships in CQL Active Record, covering BelongsTo, HasOne, HasMany, and ManyToMany. Feel free to experiment by extending your models, adding validations, or implementing more complex queries to suit your needs.

Complex Queries

CQL provides a powerful, type-safe query builder that allows you to compose sophisticated SQL queries using idiomatic Crystal code. This guide demonstrates how to build complex queries using Model.query and the CQL::Query interface.


Advanced Filtering

Combine multiple conditions, logical operators, and expressions:


Complex Joins

Join multiple tables, use aliases, and filter on joined data:


Grouping, Aggregation, and Having

Use grouping and aggregate functions for reporting and analytics:


Subqueries

Use subqueries for advanced filtering and data retrieval:


Common Table Expressions (CTEs)

Use CTEs for reusable subqueries and complex data transformations:


Pagination with Complex Queries

Combine pagination with other query methods:


Best Practices for Complex Queries

  • Use table aliases for clarity in multi-join queries.

  • Use block syntax for complex conditions to improve readability.

  • Use .select to limit columns and improve performance.

  • Use .group_by and aggregates for reporting.

  • Use subqueries and CTEs for advanced analytics and filtering.

  • Always test and inspect generated SQL with .to_sql for correctness and performance.


For more details on the query interface, see the .

AcmeDB = CQL::Schema.define(
  :acme_db,
  adapter: CQL::Adapter::Postgres,
  uri: ENV["DATABASE_URL"]
) do
  table :posts do
    primary # Defaults to :id, Int64, auto_increment: true
    text :title
    text :body
    timestamp :published_at
  end

  table :tags do
    primary
    text :name
  end

  table :post_tags do
    primary
    bigint :post_id, index: true
    bigint :tag_id, index: true
    index [:post_id, :tag_id], unique: true
  end
end
struct Post < CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :posts

  property id : Int64?
  property title : String
  property body : String
  property published_at : Time?

  # Initializing a new post
  def initialize(@title : String, @body : String, @published_at : Time? = Time.utc)
  end

  # Association: A Post has many Tags through the 'post_tags' join table.
  # `join_through` refers to the table name of the join model.
  many_to_many :tags, Tag, join_through: :post_tags
end
struct Tag < CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :tags

  property id : Int64?
  property name : String

  # Initializing a new tag
  def initialize(@name : String)
  end

  # Association: A Tag has many Posts through the 'post_tags' join table.
  many_to_many :posts, Post, join_through: :post_tags
end
struct PostTag < CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :post_tags

  property id : Int64?
  property post_id : Int64?
  property tag_id : Int64?

  # Initializing a PostTag. IDs are typically set by the association logic.
  def initialize(@post_id : Int64? = nil, @tag_id : Int64? = nil)
  end

  # Associations to the parent models
  belongs_to :post, Post, foreign_key: :post_id
  belongs_to :tag, Tag, foreign_key: :tag_id
end
# Create a new Post
post = Post.create!(title: "Crystal Language Guide", body: "This is a guide about Crystal.")

# Option 1: Create new Tags and associate them via the collection's `create` method
# This creates the Tag, saves it, and creates the PostTag join record.
tag_crystal = post.tags.create(name: "Crystal")
tag_programming = post.tags.create(name: "Programming")
puts "Post '#{post.title}' now has tags: #{post.tags.all.map(&.name).join(", ")}"

# Option 2: Add existing (persisted) Tags using `<<`
# First, ensure the tags exist or create them.
tag_tech = Tag.find_or_create_by(name: "Tech")
tag_tech.save! if !tag_tech.persisted?

post.tags << tag_tech
puts "After adding 'Tech', Post tags: #{post.tags.all.map(&.name).join(", ")}"

# Trying to add an unpersisted tag with `<<` will raise an error.
# new_unpersisted_tag = Tag.new(name: "Unsaved")
# begin
#   post.tags << new_unpersisted_tag # This would fail
# rescue e : ArgumentError
#   puts "Error adding unsaved tag: #{e.message}"
# end

# Option 3: Setting associations by an array of IDs (post.tag_ids = [...])
# This is a common pattern for updating all associations at once.
# It typically clears existing associations for this post and creates new ones.

# Ensure some tags exist to get their IDs
tag_guide = Tag.find_or_create_by(name: "Guide") { |t| t.save! }
tag_cql = Tag.find_or_create_by(name: "CQL") { |t| t.save! }

# Assuming ManyCollection supports `ids=` (verify from ManyCollection API)
# post.tag_ids = [tag_crystal.id.not_nil!, tag_guide.id.not_nil!, tag_cql.id.not_nil!]
# This functionality (direct assignment to `tag_ids=`) depends on its specific implementation in `ManyCollection`.
# If not directly available, an alternative is to clear and then add:
post.tags.clear
post.tags << tag_crystal
post.tags << tag_guide
post.tags << tag_cql
puts "After setting by IDs (clear & add), Post tags: #{post.tags.all.map(&.name).join(", ")}"
# Fetch the post
loaded_post = Post.find!(post.id.not_nil!)

puts "\nTags for post '#{loaded_post.title}':"
loaded_post.tags.all.each do |tag|
  puts "- #{tag.name}"
end

# Fetch the tag and its posts
loaded_tag_crystal = Tag.find!(tag_crystal.id.not_nil!)
puts "\nPosts for tag '#{loaded_tag_crystal.name}':"
loaded_tag_crystal.posts.all.each do |p|
  puts "- #{p.title}"
end

# Other collection methods like find_by, exists?, size, empty? also work:
tech_tag_on_post = loaded_post.tags.find_by(name: "Tech")
puts "Post has 'Tech' tag? #{!tech_tag_on_post.nil?}"
# Remove the 'Programming' tag association from the post
programming_tag = loaded_post.tags.find_by(name: "Programming")
if prog_tag = programming_tag
  if loaded_post.tags.delete(prog_tag) # Pass instance or its ID
    puts "Removed 'Programming' tag association from '#{loaded_post.title}'."
  end
end
puts "Post tags after removing 'Programming': #{loaded_post.tags.all.map(&.name).join(", ")}"

# The 'Programming' tag itself still exists in the `tags` table
still_exists_programming_tag = Tag.find_by(name: "Programming")
puts "'Programming' tag still exists globally? #{!still_exists_programming_tag.nil?}"
puts "Tags before clear for '#{loaded_post.title}': #{loaded_post.tags.size}"
loaded_post.tags.clear
puts "Tags after clear for '#{loaded_post.title}': #{loaded_post.tags.size}" # Should be 0
# Fetches all posts and their associated tags efficiently
posts_with_tags = Post.query.includes(:tags).all(Post)

posts_with_tags.each do |p|
  puts "Post: #{p.title} has tags: #{p.tags.map(&.name).join(", ") || "None"} (already loaded)"
end
# Multiple AND/OR conditions
users = User.query.where { (age >= 18) & (active == true) }
                 .where { (name.like("A%")) | (email.like("%@gmail.com")) }
                 .all(User)

# Nested conditions
admins = User.query.where { (role == "admin") & ((status == "active") | (status == "pending")) }
                   .all(User)

# IN and NOT IN
moderators = User.query.where { role.in(["admin", "moderator"]) }
                       .all(User)

excluded = User.query.where { id.not_in([1, 2, 3]) }
                     .all(User)
# Join with conditions
users_with_posts = User.query.join(Post) { users.id == posts.user_id }
                            .where { posts.published == true }
                            .all(User)

# Multiple joins with aliases
User.query.from(users: :u)
         .join({posts: :p}) { u.id == p.user_id }
         .join({comments: :c}) { p.id == c.post_id }
         .where { c.created_at > 1.week.ago }
         .all(User)
# Group by and aggregate
role_stats = User.query.select { [role, CQL.count(id).as("total")] }
                      .group_by(:role)
                      .all(User)

# Group by multiple columns
stats = Post.query.select { [user_id, CQL.count(id).as("post_count")] }
                  .group_by(:user_id)
                  .having { CQL.count(id) > 5 }
                  .all(Post)

# Aggregates: sum, avg, min, max
summary = Order.query.select { [CQL.sum(:total).as("total_sum"), CQL.avg(:total).as("avg_total")] }
                     .where { status == "paid" }
                     .all(Order)
# Users with more than 5 posts
users = User.query.where { id.in(
  Post.query.select(:user_id)
      .group_by(:user_id)
      .having { CQL.count(id) > 5 }
) }.all(User)

# Users with recent activity in posts or comments
users = User.query.where { id.in(
  Post.query.select(:user_id)
      .where { created_at > 1.week.ago }
      .union(
        Comment.query.select(:user_id)
               .where { created_at > 1.week.ago }
      )
) }.all(User)
# Users with their post and comment counts
User.query.with(:user_stats) {
  User.query.select { [id,
                      CQL.count(posts.id).as("post_count"),
                      CQL.count(comments.id).as("comment_count")] }
      .left_join(Post) { users.id == posts.user_id }
      .left_join(Comment) { users.id == comments.user_id }
      .group_by(:id)
}
.select { [users.*, user_stats.*] }
.join(:user_stats) { users.id == user_stats.id }
.all(User)
# Paginated search with sorting
page = 2
per_page = 20
users = User.query.where { name.like("%search%") }
                .order(:name)
                .offset((page - 1) * per_page)
                .limit(per_page)
                .all(User)
Querying guide

Database Migrations

Database migrations are a crucial part of managing your application's database schema in a structured and version-controlled way. CQL provides a migration system that allows you to evolve your database schema over time as your application's requirements change.

Migrations are Crystal classes that define how to apply changes (up method) and how to revert them (down method).


What are Migrations?

As your application evolves, you'll often need to:

  • Create new tables.

  • Add, remove, or modify columns in existing tables.

  • Add or remove indexes.

  • Perform other schema alterations.

Migrations allow you to script these changes in Crystal code. Each migration file typically represents a single, atomic change to the database schema and is often timestamped or versioned to ensure changes are applied in the correct order.

Benefits of using migrations:

  • Version Control: Schema changes are tracked in your project's version control system (e.g., Git) alongside your application code.

  • Collaboration: Makes it easier for teams to manage database schema changes consistently across different development environments.

  • Reproducibility: Ensures that the database schema can be recreated reliably in any environment (development, testing, production).

  • Automation: Schema changes can be applied automatically as part of deployment processes.


Defining a Migration

Migrations in CQL are Crystal classes that inherit from CQL::Migration (or a similar base class provided by the CQL framework).

  1. File Naming and Location: Migration files are typically placed in a db/migrate/ directory within your project. The filename often includes a timestamp or a sequential version number to denote the order of execution, followed by a descriptive name for the migration (e.g., db/migrate/20231027000000_create_users.cr).

  2. Migration Class Structure:

    # db/migrate/YYYYMMDDHHMMSS_create_users.cr
    # Replace YYYYMMDDHHMMSS with the actual timestamp for the migration.
    class CreateUsers < CQL::Migration
      # Optional: Define a version for this migration if not derived from filename.
      # self.version = 20231027000000_i64 # Example version
    
      # The `up` method describes the changes to apply to the database.
      def up(schema : CQL::Schema::Definition)
        # Example: Create a 'users' table
        schema.create_table :users do |t|
          # Define columns for the table
          t.primary_key :id             # Defines an auto-incrementing primary key named 'id'.
                                        # Specifics like `bigserial` for PostgreSQL might be abstracted or configurable.
          t.text :name, null: false       # A text column for user's name, cannot be null.
          t.text :email, null: false, unique: true # Email, cannot be null, must be unique.
          t.bool :active, default: false  # Boolean for active status, defaults to false.
          t.timestamps                    # Convenience method to add `created_at` and `updated_at` (both Time, nullable by default usually).
        end
    
        # Example: Adding an index separately (if not done in create_table)
        schema.add_index :users, :email, unique: true, name: "index_users_on_email_unique"
      end
    
      # The `down` method describes how to revert the changes made in the `up` method.
      def down(schema : CQL::Schema::Definition)
        # Example: Drop the 'users' table
        schema.drop_table :users
        # If you added an index separately in `up`, you might remove it here:
        # schema.remove_index :users, name: "index_users_on_email_unique" (or by column)
      end
    end

    Key Points:

    • class CreateUsers < CQL::Migration: Your migration class inherits from CQL::Migration.

    • up(schema : CQL::Schema::Definition): This method is called when applying the migration. The schema object provides methods to manipulate the database structure.

    • down(schema : CQL::Schema::Definition): This method is called when reverting (rolling back) the migration. It should undo the changes made by the up method.

    • Schema Definition API: The methods available on the schema object (e.g., create_table, add_column, drop_table, add_index) and the column definition block (e.g., t.text, t.bool, t.primary_key, t.timestamps) are specific to CQL. The exact syntax and available types (:text, :integer, :bool, :datetime, etc.) should be referenced from the CQL documentation you are using. The example above uses common conventions.


Common Migration Operations

Here are some common operations you might perform within the up and down methods of a migration:

Table Operations

  • Create Table (schema.create_table :table_name do |t| ... end):

    # up method
    schema.create_table :posts do |t|
      t.primary_key :id
      t.references :user, foreign_key: true # Creates user_id and a foreign key constraint
      t.text :title, null: false
      t.text :body
      t.datetime :published_at
      t.timestamps
    end
    # down method
    schema.drop_table :posts
  • Drop Table (schema.drop_table :table_name): Used in the down method to remove a table created in the up method.

  • Alter Table (schema.alter_table :table_name do |t| ... end): Used for adding, removing, or changing columns on an existing table.

Column Operations (within create_table or alter_table)

  • Add Column (t.add_column :column_name, :type, options... or schema.add_column :table, :column, :type, ...):

    # up method (inside alter_table)
    # schema.alter_table :users do |t|
    #  t.integer :login_count, default: 0
    # end
    # Or, if alter_table is not the direct way for add_column with CQL
    schema.add_column :users, :login_count, :integer, default: 0, null: false
    # down method
    schema.remove_column :users, :login_count
  • Remove Column (t.remove_column :column_name or schema.remove_column :table, :column): Used to drop a column.

  • Rename Column (schema.rename_column :table, :old_name, :new_name): Changes the name of an existing column.

  • Change Column (schema.change_column :table, :column, :new_type, options...): Modifies the type or other options (like null, default) of an existing column.

    # up method
    # schema.change_column :users, :email, :text, limit: 255 # Example if changing type or options

Index Operations

  • Add Index (schema.add_index :table_name, :column_name_or_columns, options...): Indexes improve query performance on frequently searched columns.

    # up method
    schema.add_index :posts, :user_id
    schema.add_index :posts, [:title, :published_at], name: "idx_posts_on_title_and_published_at"
  • Remove Index (schema.remove_index :table_name, :column_name_or_columns or name: :index_name):

    # down method
    schema.remove_index :posts, :user_id
    schema.remove_index :posts, name: "idx_posts_on_title_and_published_at"

Foreign Keys

  • Foreign keys can often be defined when creating columns (e.g., t.references :author, foreign_key: true) or added separately using methods like schema.add_foreign_key.

# Assuming t.references in create_table does this.
# If not, explicitly:
schema.add_foreign_key :posts, :users, column: :user_id, primary_key: :id

Important: The exact method names and options for schema manipulation (create_table, add_column, available data types like :text, :integer, :bool, options like null:, default:, unique:) can vary significantly between different database adapters (PostgreSQL, MySQL) and ORM/query builder implementations. Always refer to the specific CQL documentation for the version you are using to ensure you are using the correct API for schema definition.


Running Migrations

CQL will typically provide command-line tools or Rake tasks (if integrated with Rake) to manage and run your migrations.

Common migration commands (exact syntax will depend on CQL's tooling):

  • db:migrate: Applies all pending migrations (those that haven't been run yet).

    # Example placeholder command - replace with actual CQL command
    # crystal run path/to/cql/runner.cr db:migrate CONTEXT=YourDBContext
    # or if using Rake:
    # rake db:migrate
  • db:rollback: Reverts the last applied migration.

    # rake db:rollback
  • db:schema:load: Loads the schema from a schema file (e.g., db/schema.cr or db/structure.sql) into the database. This is often used to set up a new database quickly by loading the current schema state, bypassing running all migrations individually.

  • db:schema:dump: Creates or updates a schema file based on the current state of the database. This file represents the authoritative structure of your database.

  • db:reset: Typically drops the database, recreates it, and then loads the schema (or runs all migrations). Useful for resetting the database to a clean state in development.

  • Checking Migration Status: Tools to see which migrations have been applied and which are pending.

Database Context: When running migrations, you often need to specify the database context (e.g., CONTEXT=AcmeDB) if your application uses multiple databases or if the migration runner needs to know which configuration to use.


Schema File (db/schema.cr or db/structure.sql)

After migrations are run, CQL (like many ORMs) may maintain a db/schema.cr or db/structure.sql file.

  • db/schema.cr (if applicable): This would be a Crystal representation of your current database schema, generated by inspecting the database. It's often used by db:schema:load.

  • db/structure.sql: Alternatively, a raw SQL dump of the database structure. This is database-agnostic for loading but less so for inspection.

This file serves as the canonical representation of your database schema at a given point in time. It's recommended to commit this file to version control.


Migrations are a powerful tool for database schema management. Writing reversible migrations (up and down methods) is crucial, especially for rolling back changes if needed. Always test your migrations thoroughly, especially those involving data transformation or potentially destructive operations.

Scopes

Query scopes in CQL Active Record allow you to define reusable query constraints on your models. They help in making your code D.R.Y. (Don't Repeat Yourself) by encapsulating common query logic, leading to more readable and maintainable model and controller code.

The primary way to define scopes in CQL is by using the scope macro. Alternatively, scopes can also be defined as class methods.


What are Scopes?

Often, you'll find yourself writing the same query conditions repeatedly. For example:

  • Fetching all published articles.

  • Finding all active users.

  • Retrieving items created in the last 7 days.

Scopes let you give names to these common queries. A scope is essentially a pre-defined query or a piece of a query that can be easily applied and chained.

Benefits of using scopes:

  • Readability: Article.published.all is much clearer than Article.query.where(status: "published").order(published_at: :desc).all(Article) scattered throughout your codebase.

  • Reusability: Define the logic once and use it anywhere you need that specific dataset.

  • Maintainability: If the definition of "published" changes, you only need to update it in one place (the scope definition).

  • Chainability: Scopes can be chained with other scopes or standard query methods.


Defining Scopes

There are two main ways to define scopes in CQL: using the scope macro (recommended) or defining them as class methods.

1. Using the scope Macro (Recommended)

The most concise and recommended way to define scopes is by using the scope macro. This macro is provided by the CQL::ActiveRecord::Scopes module, which you should include in your model.

Syntax:

scope :scope_name, ->(optional_arguments) { query_logic }

  • :scope_name: The name of the scope (a Symbol), which will become a class method on your model.

  • ->(optional_arguments) { ... }: A proc that defines the query logic.

    • It can optionally take arguments with their types, e.g., ->(count : Int32, category : String) { ... }.

    • Inside the proc, you use standard CQL query methods like where, order, limit, etc. These methods are called on the current query context, which is typically a ChainableQuery(YourModel) instance or the model class itself if starting a new chain.

    • The proc should return a CQL::Query or a ChainableQuery(YourModel). If a raw CQL::Query is returned (e.g., by starting with query.where(...)), the scope macro intelligently wraps it in a ChainableQuery(YourModel) to ensure it remains chainable with other scopes or Active Record query methods.

Examples:

struct Article
  includes CQL::ActiveRecord::Model(Int64)
  includes CQL::ActiveRecord::Scopes # Important: Include the Scopes module
  db_context AcmeDB, :articles

  property id : Int64?
  property title : String
  property status : String # e.g., "draft", "published", "archived"
  property view_count : Int32 = 0
  property published_at : Time?

  # Scope for published articles, ordered by most recent
  scope :published, ->{ where(status: "published").order(published_at: :desc) }

  # Scope for draft articles
  scope :drafts, ->{ where(status: "draft") }
end

struct Post
  includes CQL::ActiveRecord::Model(Int64)
  includes CQL::ActiveRecord::Scopes # Important: Include the Scopes module
  db_context AcmeDB, :posts

  property created_at : Time
  property category : String
  property comment_count : Int32 = 0

  # Scope for posts created after a certain date
  scope :created_after, ->(date : Time) { where("created_at > ?", date) }

  # Scope for posts in a specific category
  scope :in_category, ->(category_name : String) { where(category: category_name) }

  # Scope for limiting results (calls 'limit' on the current query chain)
  scope :limited, ->(count : Int32) { limit(count) }

  # Scope combining other scopes/query methods
  scope :recent_in_category, ->(category_name : String, count : Int32 = 5) {
    in_category(category_name).created_after(1.week.ago).limited(count).order(comment_count: :desc)
  }
end

Using these scopes:

  • Article.published returns a ChainableQuery(Article).

  • Post.created_after(7.days.ago) returns a ChainableQuery(Post).

The scope macro generates a class method. For example, scope :published, ->{ where(status: "published") } on Article is roughly equivalent to:

# def self.published
#   # Proc is called, self inside proc refers to the current query object or class
#   scope_call_result = (->{ where(status: "published") }).call
#
#   if scope_call_result.is_a?(CQL::Query)
#     ChainableQuery(Article).new(scope_call_result)
#   else
#     scope_call_result # Assumed to be ChainableQuery(Article)
#   end
# end

This keeps your model definitions clean and focused on the query logic.

2. Defining Scopes with Class Methods (Alternative)

Alternatively, you can define scopes by creating class methods directly. This approach might be preferred for very complex logic that is hard to express in a single proc or if you prefer the explicitness of a full method definition.

When defining scopes as class methods, they should generally return a ChainableQuery(YourModel) instance to maintain chainability. The Queryable module (included via CQL::ActiveRecord::Model) provides methods like where, order, etc., that return ChainableQuery(YourModel) instances and can be used here.

Example: Basic Scopes as Class Methods

struct Article
  includes CQL::ActiveRecord::Model(Int64)
  # CQL::ActiveRecord::Scopes module is not strictly needed if only using class methods for scopes
  db_context AcmeDB, :articles

  property id : Int64?
  property title : String
  property status : String
  property published_at : Time?

  def self.published
    where(status: "published").order(published_at: :desc)
  end

  def self.drafts
    where(status: "draft")
  end
end

Example: Scopes with Arguments as Class Methods

struct Post
  includes CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :posts

  property created_at : Time
  property category : String

  def self.created_after(date : Time)
    where("created_at > ?", date)
  end

  def self.in_category(category_name : String)
    where(category: category_name)
  end

  # Note: If you name a class method scope 'limit', it might hide the
  # underlying Queryable#limit if not handled carefully. Using a different name like 'take'
  # or being explicit (e.g. self.class.query.limit(c)) might be clearer in some cases.
  def self.take(count : Int32)
    limit(count) # Calls Queryable.limit available to the class
  end
end

Scopes Returning CQL::Query (Less Common for Chaining)

While ChainableQuery(YourModel) is preferred for easy chaining (and is what the scope macro ensures), a class method scope can return a raw CQL::Query object. This is less common if you want to chain with other Active Record-style methods but might be useful for specific scenarios where you intend to pass the query object to a system expecting a base CQL::Query.

struct Product
  includes CQL::ActiveRecord::Model(Int64)
  db_context AcmeDB, :products
  property name : String?
  property stock_count : Int32 = 0

  # Scope returning a raw CQL::Query
  def self.low_stock(threshold : Int32)
    query.where("stock_count < ?", threshold) # Model.query returns a base CQL::Query
  end
end

When a scope returns CQL::Query, terminate it with methods like .all(Product) or .first(Product).


Using Scopes

Once defined (via macro or class method), scopes are called like any other class method:

# Using macro-defined scopes on Article model
published_articles = Article.published.all
puts "#{published_articles.size} published articles found."

# Using macro-defined scopes with arguments on Post model
recent_crystal_posts = Post.in_category("Crystal Lang").created_after(7.days.ago).limited(5).all
puts "#{recent_crystal_posts.size} recent Crystal Lang posts found."

# Using a class-method scope returning CQL::Query on Product model
low_stock_items = Product.low_stock(10).all(Product)
puts "#{low_stock_items.size} products with low stock."

Chaining Scopes

Scopes returning ChainableQuery(YourModel) are designed for chaining with each other and standard query methods (.where, .order, etc.):

# Chaining scopes on Article model
highly_viewed_published_articles = Article.published
  .where("view_count > ?", 1000)
  .order(view_count: :desc)
  .limited(10) # Assuming a 'limited' scope or direct .limit call
  .all

# Chaining scopes on Post model
featured_tech_posts = Post.in_category("Technology")
  .created_after(1.month.ago)
  .limited(3)
  .order(comment_count: :desc)
  .all

The ChainableQuery class often uses forward_missing_to Target (where Target is your model class). This allows class methods on your model (including those generated by the scope macro or defined manually) to be called on a ChainableQuery instance, enabling natural chaining like Article.where(...).published.


Scopes are a powerful feature for organizing your database query logic, making your application easier to read, write, and maintain. They promote the principle of keeping data logic within the model layer.