Mail (Mailer)

Will10/30/24About 4 min

Mail (Mailer)

The system provides comprehensive email sending and management capabilities, supporting real SMTP delivery and Stub test mode, with integrated async queues, database template management, and delivery logging.

Feature Overview

Module	Feature	Description
Email Sending	Plain text/HTML	Support plain + HTML multipart emails
Template Email	Tera template engine	Read templates from database, variable rendering
Send Queue	Redis async queue	Sidekiq-style Worker-based async sending
Send Log	Full lifecycle tracking	pending → success/failed state transitions
Email Template	Database management	CRUD operations, enable/disable templates
Config View	Masked display	Read SMTP config from config file, display with masking
Stub Mode	Test mode	Does not actually send, for development debugging

Database Table Structure

sys_mail_log (Email Log)

Field	Type	Description
id	i64 (Snowflake)	Primary key
recipient	String(256)	Recipient email
subject	String(256)	Email subject
content_text	String(2048)	Plain text content
content_html	Text	HTML content
from_addr	String(256)	Sender
reply_to	String(256)	Reply-to address
status	String(32)	Status: pending / success / failed
error_message	String(1024)	Error message
mail_type	String(32)	Type: plain / template
created_by	i64	Creator ID
created_at	DateTime	Created time

State Transitions:

pending  ──→  success   (sent successfully)
   │
   └──→  failed     (send failed, records error_message)

sys_mail_template (Email Template)

Field	Type	Description
id	i64 (Snowflake)	Primary key
name	String(128)	Template name
code	String(64) (Unique)	Template code (for API calls)
subject	String(256)	Email subject (supports Tera variables)
text_content	Text	Plain text template content
html_content	Text	HTML template content
description	String(512)	Template description
status	i32	Status: 0=disabled 1=enabled

API Endpoints

Path	Method	Description
`/mailer/send`	POST	Send plain text/HTML email
`/mailer/send_template`	POST	Send template email
`/mailer/config`	GET	Get mailer config (masked)
`/mailer/log_list`	GET	Get email log list
`/mailer/template/list`	GET	Get email template list
`/mailer/template/add`	POST	Add email template
`/mailer/template/edit`	PUT	Edit email template
`/mailer/template/del`	DELETE	Delete email template

Send Email Request Parameters

// POST /mailer/send
{
  "to": "user@example.com",        // Recipient (required, email format validation)
  "subject": "Email Subject",       // Subject (required)
  "text": "Plain text content",     // Plain text (required)
  "html": "<p>HTML content</p>",    // HTML content (optional)
  "from": "Display Name",           // Sender display name (optional, uses config default)
  "reply_to": "reply@example.com"   // Reply-to address (optional, email format validation)
}

Send Template Email Request Parameters

// POST /mailer/send_template
{
  "to": "user@example.com",        // Recipient (required)
  "template_code": "welcome",      // Template code (required)
  "from": "Display Name",           // Sender display name (optional)
  "reply_to": "reply@example.com", // Reply-to address (optional)
  "locals": {                       // Template variables (JSON)
    "name": "John",
    "code": "123456"
  }
}

Email Sending Architecture

Plain Text/HTML Email

Frontend Request
    │
    ▼
┌──────────────────────────┐
│  Service Layer            │
│  s_sys_mailer::send_email │
│                           │
│  1. Record pending log    │
│  2. Build Email object    │
│  3. enqueue_sync to queue │
└──────────┬───────────────┘
           │ Redis BRPOP
           ▼
┌──────────────────────────┐
│  MailerWorker             │
│  (mailer queue consumer)  │
│                           │
│  1. EmailSender init      │
│     ├─ stub=true → Stub   │
│     └─ stub=false → SMTP  │
│  2. Build lettre Message  │
│  3. Send email            │
│  4. Update log status     │
│     ├─ success            │
│     └─ failed + error     │
└──────────────────────────┘

Template Email

Frontend Request
    │
    ▼
┌──────────────────────────────────┐
│  Service Layer                    │
│  s_sys_mailer::send_template_email│
│                                   │
│  1. Read template from DB         │
│     render_from_db(template_code) │
│     ├─ Query sys_mail_template    │
│     ├─ Check template status      │
│     └─ Tera render variables      │
│  2. Record pending log            │
│  3. Build Email object            │
│  4. enqueue_sync to queue         │
└──────────┬───────────────────────┘
           │ Redis BRPOP
           ▼
┌──────────────────────────┐
│  MailerWorker (same)      │
└──────────────────────────┘

SMTP Configuration

Configure in config/development.yaml:

# Mailer Configuration
mailer:
  stub: true                    # true=test mode (no actual send), false=SMTP send
  smtp:
    enable: true
    host: smtp.163.com
    port: 465
    secure: true                # true=TLS, false=plaintext
    from: "Rust Qiluo System <noreply@163.com>"  # Default sender (display name + email)
    auth:
      user: your_email@163.com  # SMTP auth username
      password: your_password   # SMTP auth password/authorization code

Sender Handling Logic:

The from field supports display name + email format (e.g. "Rust Qiluo System <noreply@163.com>"), or display name only
The actual SMTP envelope sender address is forced to use the auth user email (auth.user)
The from passed from the frontend is also used only as a display name
This is because major SMTP servers like 163, QQ, and Gmail reject non-authenticated sender addresses

Email Template Engine

The system uses the [Tera] template engine. Templates are read from the database (sys_mail_template table) and rendered via the render_from_db function.

Database Templates (Primary Method)

Managed through the sys_mail_template table, identified by the code field:

Field	Description
name	Template name
code	Template code (unique identifier, e.g. `welcome`)
subject	Subject template (supports Tera variables)
text_content	Plain text template (optional)
html_content	HTML template (optional)
description	Template description
status	Status: 1=enabled 0=disabled

Template syntax (Jinja2 style):

{# subject #}
Welcome {{ name }}

{# text_content #}
Dear {{ name }}, your verification code is: {{ code }}

{# html_content #}
<h1>Welcome</h1>
<p>{{ name }}, your verification code is: <strong>{{ code }}</strong></p>

Example data:

INSERT INTO sys_mail_template (name, code, subject, text_content, html_content, status)
VALUES ('Welcome Email', 'welcome', 'Welcome {{ name }}', 'Verification code: {{ code }}', '<p>Verification code: {{ code }}</p>', 1);

Usage:

POST /mailer/send_template
{
  "to": "user@example.com",
  "template_code": "welcome",
  "locals": { "name": "John", "code": "123456" }
}

Rendering Flow:

Query sys_mail_template table by template_code
Check template status (return "disabled" error if status != 1)
Render locals variables into subject / text_content / html_content using Tera engine
Return rendered Content { subject, text, html }

File Templates (Deprecated)

⚠️ File template method (src/data/mailers/ directory) has been removed. All templates are managed through the database, supporting online CRUD and enable/disable.

Async Queue & Worker

Email sending is implemented based on Redis async queue with core components:

Component	Description
`MailerWorker`	Email queue consumer, processes `mailer` queue
`Processor`	Worker scheduler, BRPOP polling queue
`UnitOfWork`	Task unit, manages task lifecycle
`Scheduled`	Scheduled/retry scheduler
`EmailSender`	Email sender, wraps SMTP/Stub transport

Queue Configuration:

workers:
  queues:
    - default
    - mailer        # Dedicated email queue
  num_workers: 2    # Worker concurrency

Failure Retry Mechanism:

Exponential backoff: 5s → 10s → 20s → 40s → ... → max 3600s
No more retries after exceeding max retry count, error is logged

Code Examples

Send Email (Enqueue)

use crate::worker::mailer::{Email, MailerWorker};
let email = Email {
  from: None,
  to: "user@example.com".into(),
  reply_to: None,
  subject: "Hello".into(),
  text: "Plain text content".into(),
  html: "<b>HTML content</b>".into(),
};
let _ = MailerWorker::enqueue_async(email).await;

Send Template Email

use crate::worker::mailer::{Args, mail_template};
let _ = mail_template(
  "welcome".into(),
  Args {
    from: None,
    to: "user@example.com".into(),
    reply_to: None,
    locals: serde_json::json!({ "username": "Alice", "link": "https://example.com" }),
  }
).await;

Frontend Pages

Page Path	Feature
`/Monitor/Email`	Email management (config/send/template/log)
`/Test/Email`	Email test (simplified send page)

Email Management Page contains 4 tabs:

Config Info — Display SMTP config (masked), Stub mode status
Send Email — Plain text/HTML email send form
Template Email — Select template + fill variables to send
Send Log — Email log list, filter by recipient/subject/status

Technical Notes

Snowflake ID

All email module tables use Snowflake ID instead of auto-increment ID:

Distributed uniqueness: No ID conflicts in multi-instance deployments
Non-guessable: Avoids security risk of data traversal via ID
Time-ordered: ID itself contains time information, naturally ordered

UTF-8 Charset Fix

Email-related tables have MySQL charset fixed via Migration:

ALTER TABLE sys_mail_log CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE sys_mail_template CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

SMTP Sender Security Policy

To prevent major email service providers (163, QQ, Gmail, etc.) from rejecting non-authenticated sender addresses, the system implements the following policies:

The from field in config supports display name + email format (e.g. "Qiluo Tech <noreply@163.com>"), or display name only
The actual SMTP envelope sender is forced to use auth.user
The from passed from the frontend is used only as a display name, the email address will be replaced
If auth.user is not configured, from must be a complete email format