Day 9: Contact Search API: Filtering Contacts by Name

Contact Search API: Filtering Contacts by Name

Welcome back, engineers! Today, we're tackling a seemingly straightforward feature that holds hidden complexities and scalability challenges: building a Contact Search API. Specifically, we'll focus on filtering contacts by name. While it might sound simple – just a database query, right? – the path from a local MVP to a hyperscale system handling 100 million requests per second demands a deeper understanding of search mechanics, indexing, and API design.

This isn't just about writing code; it's about understanding the implications of that code when your CRM grows from a handful of users to millions, and your contact database explodes from hundreds to hundreds of millions.

Agenda for Lesson 9:

1. The "Why" Behind Efficient Search: Understanding the business and technical criticality.

2. Core Concepts: Simple Filtering & Its Limits: Implementing a basic name filter and immediately seeing its scalability cracks.

3. Architecting for Future Scale: Introducing the idea of indexing and dedicated search solutions.

4. Hands-on Build-Along: Implementing our GET /contacts endpoint with name filtering.

5. Assignment & Solution Hints: Your next steps to solidify this knowledge.

The "Why" Behind Efficient Search: More Than Just a Feature

Imagine a sales rep trying to find "John Doe" among 50 million contacts. If your search takes 10 seconds, that's 10 seconds of lost productivity, 10 seconds of frustration. Multiply that by thousands of reps, and you have a significant operational bottleneck.

In a hyperscale CRM:

• User Experience (UX) is King: Instantaneous search results are non-negotiable for a smooth, productive user experience. Latency kills productivity.

• System Load: Inefficient queries can bring your database to its knees, impacting all other CRM operations.

• Data Volume: As your CRM scales, the sheer volume of contact data makes naive search approaches catastrophic.

• Advanced Needs: Users will eventually want partial matches, phonetic searches, fuzzy matching, and searching across multiple fields. We're laying the groundwork for that journey.

Today, we'll start simple, but with an eye toward these future challenges.

Core Concepts: Filtering Contacts by Name

System Design Concept: Progressive Search Architecture

Our strategy for search will be progressive. We start with the simplest, most direct approach, but immediately identify its weaknesses and mentally prepare for the next evolutionary step. This is a common pattern in big tech: build minimum viable, then iterate based on real-world constraints.

Initial Architecture:
Our GET /contacts endpoint will interact directly with our PostgreSQL database. When a name query parameter is provided, we'll use a LIKE clause in our SQL query.

Control Flow:

1. A user or another service sends an HTTP GET request to /contacts?name=John.

2. Our API Gateway (or load balancer) routes the request to our CRM Backend Service.

3. The Backend Service's handler extracts the name query parameter.

4. It calls the repository layer, passing the name.

5. The repository constructs a SQL query like SELECT * FROM contacts WHERE name LIKE $1.

6. The database executes the query and returns matching contacts.

7. The repository maps the database rows to Contact structs.

8. The handler formats these structs into a JSON response.

9. The Backend Service sends the JSON response back to the client.

Data Flow:

Client (HTTP GET /contacts?name=...) -> Backend Service (extracts 'name') -> Repository (SQL query with LIKE) -> Database (returns rows) -> Repository (maps to structs) -> Backend Service (JSON response) -> Client

State Changes:

For a read-only search operation, the system's state doesn't change. However, the state of the request processing evolves: Request Received -> Parameter Parsed -> DB Query Executed -> Results Formatted -> Response Sent. The underlying contact data in the database remains the source of truth.

The Problem with LIKE '%name%' at Scale

When you write SELECT * FROM contacts WHERE name LIKE '%John%';, your database often has to perform a full table scan. This means it looks at every single row in your contacts table to find matches. For 100 contacts, it's trivial. For 100 million contacts, it's a disaster.

Insight: SQL LIKE queries with a leading wildcard (%) generally cannot leverage standard B-tree indexes, making them incredibly slow on large datasets. If you could search LIKE 'John%' (no leading wildcard), an index on the name column would be very efficient. But users rarely search just prefixes.

Real-world Application: This is precisely why big tech companies use specialized full-text search engines like Elasticsearch or Apache Solr for search functionality. These systems are designed to index vast amounts of text data and provide near-instantaneous search results, handling partial matches, relevance scoring, and complex queries much more efficiently than a relational database's LIKE operator. We'll explore these in future lessons, but for now, understand the limitation.

For our current lesson, we'll implement the basic LIKE '%name%' but be acutely aware of its limitations and the need for a future upgrade path. This pragmatic approach allows us to deliver functionality quickly while acknowledging technical debt for scale.

Component Architecture

Our CRM backend service, written in Go, will expose a new API endpoint. We'll reuse our existing Contact model and repository.

+------------------+     +------------------------+     +------------------+
|      Client      |  | CRM Backend Service    |  |   PostgreSQL DB  |
| (e.g., cURL/UI)  |     |   (Go Lang)            |     |   (Contacts Table)|
|                  |     | - HTTP Server          |     |                  |
| GET /contacts?name= |   | - Contact Handler      |     | - Store Contacts |
|                  |     | - Contact Repository   |     | - Execute Queries|
+------------------+     +------------------------+     +------------------+
                             ^
                             |
                             | (Future: Dedicated Search Service - Elasticsearch/Solr)

Hands-on Build-Along: Implementing the Search API

We'll extend our existing Go backend to add the search functionality.

1. Update repository.go

Add a new method to fetch contacts by name.

// backend/repository.go
package main

import (
	"database/sql"
	"fmt"
	"log"
)

// Contact represents a contact in the CRM system
type Contact struct {
	ID        string `json:"id"`
	FirstName string `json:"first_name"`
	LastName  string `json:"last_name"`
	Email     string `json:"email"`
	Phone     string `json:"phone"`
}

// ContactRepository handles database operations for contacts
type ContactRepository struct {
	db *sql.DB
}

// NewContactRepository creates a new ContactRepository
func NewContactRepository(db *sql.DB) *ContactRepository {
	return &ContactRepository{db: db}
}

// CreateContact inserts a new contact into the database
func (r *ContactRepository) CreateContact(contact Contact) error {
	query := `INSERT INTO contacts (id, first_name, last_name, email, phone) VALUES ($1, $2, $3, $4, $5)`
	_, err := r.db.Exec(query, contact.ID, contact.FirstName, contact.LastName, contact.Email, contact.Phone)
	if err != nil {
		log.Printf("Error creating contact: %v", err)
		return fmt.Errorf("failed to create contact: %w", err)
	}
	return nil
}

// GetContactsByName fetches contacts filtered by first_name or last_name (case-insensitive, partial match)
func (r *ContactRepository) GetContactsByName(name string) ([]Contact, error) {
	searchQuery := "%" + name + "%" // Add wildcards for partial match
	query := `SELECT id, first_name, last_name, email, phone FROM contacts 
	          WHERE LOWER(first_name) LIKE LOWER($1) OR LOWER(last_name) LIKE LOWER($1)`
	
	rows, err := r.db.Query(query, searchQuery)
	if err != nil {
		log.Printf("Error querying contacts by name '%s': %v", name, err)
		return nil, fmt.Errorf("failed to query contacts by name: %w", err)
	}
	defer rows.Close()

	var contacts []Contact
	for rows.Next() {
		var contact Contact
		if err := rows.Scan(&contact.ID, &contact.FirstName, &contact.LastName, &contact.Email, &contact.Phone); err != nil {
			log.Printf("Error scanning contact row: %v", err)
			return nil, fmt.Errorf("failed to scan contact row: %w", err)
		}
		contacts = append(contacts, contact)
	}

	if err = rows.Err(); err != nil {
		log.Printf("Error during rows iteration: %v", err)
		return nil, fmt.Errorf("error during rows iteration: %w", err)
	}

	return contacts, nil
}

// GetAllContacts fetches all contacts from the database
func (r *ContactRepository) GetAllContacts() ([]Contact, error) {
	query := `SELECT id, first_name, last_name, email, phone FROM contacts`
	rows, err := r.db.Query(query)
	if err != nil {
		log.Printf("Error querying all contacts: %v", err)
		return nil, fmt.Errorf("failed to query all contacts: %w", err)
	}
	defer rows.Close()

	var contacts []Contact
	for rows.Next() {
		var contact Contact
		if err := rows.Scan(&contact.ID, &contact.FirstName, &contact.LastName, &contact.Email, &contact.Phone); err != nil {
			log.Printf("Error scanning contact row: %v", err)
			return nil, fmt.Errorf("failed to scan contact row: %w", err)
		}
		contacts = append(contacts, contact)
	}

	if err = rows.Err(); err != nil {
		log.Printf("Error during rows iteration: %v", err)
		return nil, fmt.Errorf("error during rows iteration: %w", err)
	}

	return contacts, nil
}

Insight: Notice LOWER(first_name) LIKE LOWER($1). This makes our search case-insensitive, which is a common user expectation. Using $1 for the search term is crucial for preventing SQL injection vulnerabilities. Never concatenate user input directly into SQL queries!

2. Update handlers.go

Modify the GetContacts handler to check for the name query parameter.

// backend/handlers.go
package main

import (
	"encoding/json"
	"net/http"
	"log"
)

// ContactHandler handles HTTP requests related to contacts
type ContactHandler struct {
	repo *ContactRepository
}

// NewContactHandler creates a new ContactHandler
func NewContactHandler(repo *ContactRepository) *ContactHandler {
	return &ContactHandler{repo: repo}
}

// CreateContact handles POST requests to create a new contact
func (h *ContactHandler) CreateContact(w http.ResponseWriter, r *http.Request) {
	var contact Contact
	if err := json.NewDecoder(r.Body).Decode(&contact); err != nil {
		http.Error(w, "Invalid request body", http.StatusBadRequest)
		return
	}

	// For simplicity, generate a UUID for the contact ID here.
	// In a real system, this might be done in a service layer or even by the DB.
	contact.ID = generateUUID() 

	if err := h.repo.CreateContact(contact); err != nil {
		log.Printf("Failed to create contact: %v", err)
		http.Error(w, "Failed to create contact", http.StatusInternalServerError)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusCreated)
	json.NewEncoder(w).Encode(contact)
}

// GetContacts handles GET requests to retrieve contacts.
// It supports filtering by name via query parameter: /contacts?name=John
func (h *ContactHandler) GetContacts(w http.ResponseWriter, r *http.Request) {
	nameFilter := r.URL.Query().Get("name")

	var contacts []Contact
	var err error

	if nameFilter != "" {
		contacts, err = h.repo.GetContactsByName(nameFilter)
	} else {
		contacts, err = h.repo.GetAllContacts()
	}

	if err != nil {
		log.Printf("Failed to retrieve contacts: %v", err)
		http.Error(w, "Failed to retrieve contacts", http.StatusInternalServerError)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(contacts)
}

Insight: The r.URL.Query().Get("name") method is how you safely extract query parameters from the URL. If name is present, we use our new search method; otherwise, we fetch all contacts (which itself will become a performance bottleneck at scale – another future optimization!).

3. Update main.go

Ensure your main.go sets up the routes correctly.

// backend/main.go
package main

import (
	"database/sql"
	"fmt"
	"log"
	"net/http"
	"os"
	"time"

	"github.com/gorilla/mux"
	_ "github.com/lib/pq" // PostgreSQL driver
	"github.com/google/uuid" // For generating UUIDs
)

// generateUUID generates a new UUID
func generateUUID() string {
	return uuid.New().String()
}

func main() {
	// Database connection setup
	dbHost := os.Getenv("DB_HOST")
	dbPort := os.Getenv("DB_PORT")
	dbUser := os.Getenv("DB_USER")
	dbPassword := os.Getenv("DB_PASSWORD")
	dbName := os.Getenv("DB_NAME")

	connStr := fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=disable",
		dbHost, dbPort, dbUser, dbPassword, dbName)

	db, err := sql.Open("postgres", connStr)
	if err != nil {
		log.Fatalf("Failed to open database connection: %v", err)
	}
	defer db.Close()

	// Ping the database to ensure connection is established
	for i := 0; i < 5; i++ {
		err = db.Ping()
		if err == nil {
			log.Println("Successfully connected to the database!")
			break
		}
		log.Printf("Waiting for database to be ready (attempt %d): %v", i+1, err)
		time.Sleep(2 * time.Second)
	}
	if err != nil {
		log.Fatalf("Failed to connect to database after multiple retries: %v", err)
	}

	// Initialize repository and handler
	contactRepo := NewContactRepository(db)
	contactHandler := NewContactHandler(contactRepo)

	// Setup router
	router := mux.NewRouter()

	// Contact routes
	router.HandleFunc("/contacts", contactHandler.CreateContact).Methods("POST")
	router.HandleFunc("/contacts", contactHandler.GetContacts).Methods("GET") // Handles both /contacts and /contacts?name=...

	// Start server
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080" // Default port
	}
	log.Printf("Server starting on port %s...", port)
	log.Fatal(http.ListenAndServe(":"+port, router))
}

Insight: We've integrated the GetContacts handler to intelligently serve both unfiltered and name-filtered requests based on the presence of the name query parameter. This is a clean way to handle variations of a resource retrieval.

Assignment: Beyond Basic Filtering

Your mission, should you choose to accept it, is to enhance our search capabilities.

Assignment Steps:

1. Add a phone filter: Modify the GetContacts handler and ContactRepository to allow searching by phone number as well. The API should support /contacts?phone=+1-555-123-4567.

2. Combine filters: Make the API support both name and phone filters simultaneously. If both are present, contacts should match both criteria. Example: /contacts?name=John&phone=+1-555-123-4567.

3. Refine the GetAllContacts fallback: Currently, if no filters are present, GetAllContacts() fetches all contacts. For a large system, this is unsustainable. Modify GetAllContacts (or the GetContacts handler) to implement pagination (e.g., _limit and _offset query parameters) as a temporary measure. Don't implement it yet, just note where and how you would add it. This is a thought exercise for now, as full pagination will be a separate lesson.

Solution Hints:

1. Add phone filter:

• In handlers.go, retrieve phoneFilter := r.URL.Query().Get("phone").

• In repository.go, create a new method GetContactsByPhone(phone string) ([]Contact, error) similar to GetContactsByName.

• Update the GetContacts handler to check phoneFilter and call the new repository method.

1. Combine filters:

• The GetContacts handler will need more complex logic. If both nameFilter and phoneFilter are present, you'll need a new repository method like GetContactsByNameAndPhone(name, phone string).

• The SQL query in this new method will use WHERE (LOWER(first_name) LIKE LOWER($1) OR LOWER(last_name) LIKE LOWER($1)) AND LOWER(phone) LIKE LOWER($2). Remember to use parameterized queries!

1. Pagination thought exercise:

• Consider adding _limit and _offset query parameters.

• The GetAllContacts SQL query would become SELECT ... FROM contacts LIMIT $1 OFFSET $2.

• The handler would parse these parameters (with default values) and pass them to the repository. This is vital for any API that returns lists of resources.

This lesson equips you with a functional search API and, more importantly, the critical foresight to understand its limitations and future scaling needs. You're now building not just features, but a resilient, scalable system.