Build Your Own Mailchimp in Golang: Concurrent Email Campaign Dispatcher

This chapter guides you through building a high-performance, concurrent bulk email campaign dispatcher (an MVP of a Mailchimp clone) in Golang. By leveraging Go's native concurrency primitives—Go routines and channels—this tool achieves maximum throughput with minimal CPU and memory overhead, offering a cost-effective alternative to expensive commercial email marketing platforms.

1. The Problem & The Solution (Why Golang?)

The Problem: High Cost of Bulk Email Marketing

For organizations with large student or subscriber lists, commercial email marketing tools (like Mailchimp, Sendgrid, or ActiveCampaign) become prohibitively expensive as list sizes scale. These platforms charge steep premiums based on subscriber count rather than actual compute resources used.

The Solution: A Custom Dispatcher in Go

Building a custom email dispatcher allows you to control the delivery pipeline. Golang is the ideal language for this task due to its:

• Lightweight Concurrency Model: Go routines consume only a few kilobytes of memory compared to OS threads, allowing you to run thousands of concurrent workers on a single cheap server.
• Resource Efficiency: High CPU and RAM efficiency ensures that bulk email campaigns can be dispatched rapidly without crashing the host system.
• Native Networking Capabilities: Go's standard library provides robust, built-in packages for SMTP communication (net/smtp) and templating (html/template).

2. System Architecture & Core Mental Model

The email dispatcher is designed around a decoupled, asynchronous architecture. Instead of reading a contact list and sending emails sequentially (which is slow and blocking), we separate the data-loading process from the delivery process.

Architectural Flow

Key Components

1. Data Source: A CSV file (emails.csv) containing recipient details (Name, Email).
2. Producer: A Go routine that reads the CSV file, parses the records, and feeds them into a communication channel.
3. Channel: A thread-safe queue used to pass recipient data from the producer to the consumers.
4. Consumers (Workers): Multiple concurrent Go routines running in the background. They pull recipient data from the channel, compile the email template, and dispatch it via SMTP.
5. SMTP Server: The delivery agent. For local development and testing, we use Mailpit; in production, this can be swapped with services like Amazon SES, Resend, or SendGrid.

3. The Producer-Consumer Pattern & Unbuffered Channels

To coordinate data flow between the file reader and the email senders, we use the Producer-Consumer Pattern connected via an Unbuffered Channel.

Real-Life Analogy: Playing Catch

Imagine you and a friend are playing catch with a baseball:

• The Producer is you (throwing the ball).
• The Consumer is your friend (catching the ball).
• The Unbuffered Channel is the air between you.

If your friend is not looking or is unready, you cannot throw the ball—you must wait. The ball cannot hang suspended in mid-air (it cannot be stored in the channel because an unbuffered channel has a capacity of $0$). The moment your friend is ready, you throw, and they catch it instantly.

In Go, this means the Sender blocks until a Receiver is ready to read from the channel. This prevents the producer from overloading system memory with millions of records if the workers are slow.

Channel Strategies Comparison

4. Setting Up the Golang Project & Environment

Step 1: Initialize the Module

Create a new directory and initialize a Go module using standard naming conventions:

mkdir email-dispatcher
cd email-dispatcher
go mod init github.com/yourusername/email-dispatcher

Step 2: Verify Go Installation

Ensure you are running a modern version of Go (preferably 1.22+):

go version

Step 3: Create the Project Structure

Create the following files in your project root:

• main.go: Application entry point and orchestrator.
• producer.go: Contains CSV parsing and channel feeding logic.
• consumer.go: Contains worker pool and SMTP dispatch logic.
• email.tmpl: HTML/Text template for the email body.
• emails.csv: Mock contact list.

5. Implementing the Producer (CSV Parser & Channel Sender)

The producer is responsible for opening the CSV file, skipping the header row, parsing each record into a structured format, and sending it to the channel.

Recipient Struct Definition

In main.go, define the custom data structure:

package main

// Recipient represents a single email campaign target.
// Fields are capitalized to make them public/exported.
type Recipient struct {
	Name  string
	Email string
}

Producer Implementation (producer.go)

Create producer.go and add the following code:

package main

import (
	"encoding/csv"
	"fmt"
	"os"
)

// loadRecipients opens a CSV file, parses records, and sends them to a write-only channel.
func loadRecipients(filePath string, ch chan<- Recipient) error {
	// Open the CSV file
	file, err := os.Open(filePath)
	if err != nil {
		return fmt.Errorf("failed to open file: %w", err)
	}

	// Ensure the file is closed when this function exits
	defer file.Close()

	// Ensure the channel is closed when production is complete
	defer close(ch)

	reader := csv.NewReader(file)
	records, err := reader.ReadAll()
	if err != nil {
		return fmt.Errorf("failed to read CSV records: %w", err)
	}

	// Loop through records, skipping the header row (index 0)
	for _, record := range records[1:] {
		if len(record) < 2 {
			continue // Skip malformed rows
		}

		recipient := Recipient{
			Name:  record[0],
			Email: record[1],
		}

		// Send recipient to channel (blocks until a worker is ready to receive)
		ch <- recipient
	}

	return nil
}

Note on defer close(ch): Closing the channel is critical. It signals to all listening workers that no more data is coming, allowing their loops to terminate gracefully instead of waiting indefinitely (which causes a deadlock).

6. Implementing the Consumer (Concurrent Workers & SMTP Dispatcher)

Consumers run as independent Go routines. Each worker loops over the channel, receives a recipient, formats the email, and sends it via SMTP.

Consumer Implementation (consumer.go)

package main

import (
	"fmt"
	"log"
	"net/smtp"
	"sync"
	"time"
)

// emailWorker processes incoming recipients from the channel and dispatches emails.
func emailWorker(id int, ch <-chan Recipient, wg *sync.WaitGroup) {
	// Signal to the WaitGroup that this worker has finished when the function exits
	defer wg.Done()

	smtpHost := "localhost"
	smtpPort := "1025"
	fromEmail := "rakesh@codersgan.com"

	// Range loop automatically terminates when the channel is closed
	for recipient := range ch {
		fmt.Printf("[Worker %d] Preparing email for %s (%s)\n", id, recipient.Name, recipient.Email)

		// Compile the email template
		messageBody, err := executeTemplate(recipient)
		if err != nil {
			log.Printf("[Worker %d] Error parsing template for %s: %v", id, recipient.Email, err)
			continue // Skip to next recipient on failure
		}

		// Construct the raw SMTP message payload
		addr := smtpHost + ":" + smtpPort
		toHeader := fmt.Sprintf("To: %s\r\n", recipient.Email)
		subjectHeader := "Subject: Exclusive Updates from Coders Gyan!\r\n"
		mimeHeader := "MIME-Version: 1.0\r\nContent-Type: text/html; charset=UTF-8\r\n\r\n"
		
		msg := []byte(toHeader + subjectHeader + mimeHeader + messageBody)

		// Send the email (No authentication required for local Mailpit testing)
		err = smtp.SendMail(addr, nil, fromEmail, []string{recipient.Email}, msg)
		if err != nil {
			log.Printf("[Worker %d] Failed to send email to %s: %v", id, recipient.Email, err)
			continue
		}

		fmt.Printf("[Worker %d] Successfully sent email to %s\n", id, recipient.Email)

		// Rate limiting: Sleep briefly to avoid overwhelming the SMTP server
		time.Sleep(50 * time.Millisecond)
	}
}

7. Synchronization & Preventing Deadlocks

When writing concurrent Go programs, you must manage the lifecycle of Go routines to prevent Deadlocks and Premature Termination.

The Deadlock Pitfall

If the main Go routine finishes execution, the entire program exits immediately, killing any background Go routines before they can finish their work. Conversely, if a Go routine waits on a channel that is never closed, or if the main thread waits on a synchronization primitive that is never released, Go throws a runtime panic: fatal error: all goroutines are asleep - deadlock!.

The Solution: sync.WaitGroup

A sync.WaitGroup acts as a concurrent counter.

1. Increment: Call wg.Add(1) before spawning a worker Go routine.
2. Decrement: Call wg.Done() (usually deferred) inside the worker when it finishes.
3. Block: Call wg.Wait() in the main thread to block execution until the counter returns to $0$.

8. Setting Up Mailpit for Local SMTP Testing

To test bulk email dispatching without sending real spam or getting blacklisted, use Mailpit, a lightweight SMTP testing tool with a built-in web interface.

Running Mailpit with Docker

Run the following command in your terminal to start Mailpit:

docker run -d \
  --name mailpit \
  -p 1025:1025 \
  -p 8025:8025 \
  axllent/mailpit

• Port 1025: The SMTP port where our Go application sends emails.
• Port 8025: The HTTP port where you can view the web-based email inbox (http://localhost:8025).

9. Adding Dynamic HTML/Text Templating

Hardcoding email bodies inside Go code is unmaintainable. Go's html/template package allows you to separate presentation from logic.

Step 1: Create the Template File (email.tmpl)

Create a file named email.tmpl in your project root:

<!DOCTYPE html>
<html>
<head>
    <style>
        body { font-family: Arial, sans-serif; line-height: 1.6; color: #333; }
        .container { padding: 20px; border: 1px solid #eee; border-radius: 5px; max-width: 600px; }
        .footer { margin-top: 20px; font-size: 0.8em; color: #777; }
    </style>
</head>
<body>
    <div class="container">
        <h2>Hello {{.Name}},</h2>
        <p>We have some exciting news to share with you! We are launching our new advanced backend engineering curriculum this month.</p>
        <p>Stay tuned for more updates and early-bird discounts.</p>
        <div class="footer">
            <p>Regards,<br><strong>The Coders Gyan Campaign Team</strong></p>
        </div>
    </div>
</body>
</html>

Step 2: Implement Template Parsing in Go

Add the template execution helper function to your codebase (can be placed in consumer.go or a separate utility file):

package main

import (
	"bytes"
	"fmt"
	"html/template"
)

// executeTemplate parses the template file and injects recipient data.
func executeTemplate(recipient Recipient) (string, error) {
	// Parse the template file
	tmpl, err := template.ParseFiles("email.tmpl")
	if err != nil {
		return "", fmt.Errorf("failed to parse template file: %w", err)
	}

	// Use a bytes.Buffer to capture the output of template execution
	var tpl bytes.Buffer
	err = tmpl.Execute(&tpl, recipient)
	if err != nil {
		return "", fmt.Errorf("failed to execute template: %w", err)
	}

	return tpl.String(), nil
}

10. Complete Codebase Integration Walkthrough

Here is how all the components tie together in main.go.

Orchestrator Implementation (main.go)

package main

import (
	"fmt"
	"log"
	"sync"
	"time"
)

func main() {
	startTime := time.Now()
	fmt.Println("=== Starting Email Dispatcher MVP ===")

	// 1. Create an unbuffered channel for Recipient data
	recipientChan := make(chan Recipient)

	// 2. Initialize a WaitGroup to track active workers
	var wg sync.WaitGroup

	// 3. Define the number of concurrent workers
	workerCount := 5

	// 4. Spawn the worker pool (Consumers)
	for i := 1; i <= workerCount; i++ {
		wg.Add(1)
		go emailWorker(i, recipientChan, &wg)
	}

	// 5. Spawn the data loader (Producer) in a separate Go routine
	// This prevents the main thread from blocking on channel writes
	go func() {
		err := loadRecipients("emails.csv", recipientChan)
		if err != nil {
			log.Fatalf("Producer error: %v", err)
		}
	}()

	// 6. Block main execution until all workers have finished processing
	wg.Wait()

	fmt.Printf("=== Campaign Completed Successfully in %v ===\n", time.Since(startTime))
}

Sample Data (emails.csv)

Create a mock CSV file named emails.csv in your project root:

Name,Email
Aarav Sharma,aarav@example.com
Ananya Iyer,ananya@example.com
Vihaan Patel,vihaan@example.com
Diya Sen,diya@example.com
Kabir Gupta,kabir@example.com

Running the Application

Execute the program by running:

go run .

Output:

=== Starting Email Dispatcher MVP ===
[Worker 1] Preparing email for Aarav Sharma (aarav@example.com)
[Worker 2] Preparing email for Ananya Iyer (ananya@example.com)
[Worker 3] Preparing email for Vihaan Patel (vihaan@example.com)
[Worker 1] Successfully sent email to aarav@example.com
[Worker 4] Preparing email for Diya Sen (diya@example.com)
...
=== Campaign Completed Successfully in 254.32ms ===

11. Production Edge Cases, Pitfalls, & Future Enhancements

While this MVP is highly functional, running a production-grade email campaign tool requires addressing several real-world operational challenges.

1. Rate Limiting and IP Warmups

Sending thousands of emails instantly will cause receiving servers (Gmail, Outlook) to flag your IP as spam.

• Mitigation: Implement token bucket rate-limiting algorithms to restrict dispatches to a specific rate (e.g., 50 emails per minute) and gradually warm up new IP addresses.

2. Error Handling & Dead Letter Queues (DLQ)

If an SMTP server fails temporarily, simply logging the error and continuing results in lost emails.

• Mitigation: Implement a retry mechanism with exponential backoff. If an email fails after 3 attempts, push it to a Dead Letter Queue (DLQ) in a database (e.g., PostgreSQL or Redis) for manual review or automatic retry later.

3. Database Integration

Reading from a CSV is not scalable for dynamic lists or segmentations.

• Mitigation: Replace the CSV reader in the producer with SQL queries that fetch users based on specific tags or segments:
  SQL

  Copy

  SELECT name, email FROM users WHERE segment = 'active_students';
  

4. Future Feature Roadmap

To transform this MVP into a fully-fledged SaaS product, consider implementing:

• Campaign Scheduling: Store campaigns in a database with a scheduled_at timestamp and use a cron-like runner in Go to trigger them.
• Tracking Pixels: Inject a transparent $1\times1$ pixel image into the HTML template to track email open rates.
• Unsubscribe Headers: Add standard List-Unsubscribe headers and handle opt-out requests automatically.

12. Summary and Key Takeaways

• Go Routines enable massive concurrency with minimal resource consumption, making Go the premier choice for high-throughput network applications like email dispatchers.
• Unbuffered Channels act as synchronous, zero-capacity queues that block the sender until a receiver is ready, naturally preventing memory bloat.
• sync.WaitGroup is essential for coordinating Go routine lifecycles, ensuring the main program waits for all background tasks to complete before exiting.
• Closing Channels is a mandatory step to signal to consumer loops that data production has finished, preventing permanent blocking and deadlocks.
• Mailpit provides a safe, sandboxed local SMTP environment that accelerates development and testing without risking domain reputation.