User Tools

Site Tools


gitlab_ci_cd_documentation:cotrav-platform

This is an old revision of the document!


Table of Contents

GitLab CI/CD Documentation: Cotrav-Platform

1. Project Overview

The Cotrav-Platform is built using a Microservices Architecture. This document provides the technical details of our automation pipeline, which handles building, testing, and deploying each service independently.

Cotrav Platform is a monorepo-based system that manages multiple backend services, shared libraries, and frontend applications in a single repository.

The repository is hosted on GitLab and uses GitLab CI/CD for automated build and deployment.

Main components of the platform:

  • Backend Microservices
  • Frontend UI
  • Shared Libraries
  • CI/CD Pipeline
  • Docker-based Deployment
  • NGINX Reverse Proxy
2. Infrastructure & Tools

We use the following stack to manage our operations:

Version Control: GitLab

CI/CD Engine: GitLab CI/CD (defined in .gitlab-ci.yml)

Containerization: Docker

Registry: GitLab Container Registry

Target Environment: etc

 Getting Started & Repository Setup

2. Repository Structure (Monorepo)

cotrav-platform
│
├── Cotrav_Services # Backend microservices
│
├── Cotrav_UI # Frontend application
│
├── shared # Shared packages and utilities
│
├── gitlab/templates # CI/CD reusable templates
│
├── .gitlab-ci.yml # CI/CD pipeline configuration
│
├── package-lock.json
│
└── README.md
  • Step 1: Open your terminal and navigate to your project folder.
  • Step 2: Add the remote origin and push your code using these commands:

Add your files

cd existing_repo
git remote add origin https://gitlab.com/cotrav-tech/cotrav-platform.git
git branch -M main
git push -uf origin main''

The pipeline configuration is defined in the .gitlab-ci.yml file located in the root of the repository.

This pipeline is optimized for a monorepo architecture, where multiple services exist in a single repository. The pipeline automatically detects which services have changed and triggers CI/CD only for those services.

The pipeline runs using GitLab Runner with the runner tag:

Root - ''gitlab-ci.yml ''

GitLab runner— docker-runner01

Pipeline Stages

The pipeline consists of three main stages:

detect trigger post-deploy-cleanup

Each stage has a specific responsibility.

Stage 1 — Detect Changes

Job Name:

detect_changes

Purpose:

Detect which services or components have changed in the repository.

The pipeline compares the current commit with the previous commit using:

git diff

It then determines:

  • Which backend services have changed
  • Whether the frontend has changed
  • Whether shared packages changed

If shared packages change, all services are triggered.

Example Logic

FILES_CHANGED=$(git diff –name-only $BEFORE_SHA $CI_COMMIT_SHA)

The script then checks changes inside:

Cotrav_Services/services/
Cotrav_Services/environments/

If changes are detected, the pipeline dynamically generates a file:

trigger-pipeline.yml

This file contains the list of services whose pipelines should be triggered.

Artifact Generated

trigger-pipeline.yml

This artifact is passed to the next stage.


Job Name:

Stage 2 — Trigger Service Pipelines

trigger_services

Purpose:

Trigger CI/CD pipelines for the services that changed.

This stage reads the generated file:

trigger-pipeline.yml

and dynamically includes service-specific pipeline configurations.

Example trigger:

trigger:
include:
- local: Cotrav_Services/services/auth-service/.gitlab-ci.yml  exmple ... flight , Hotel etc.

Each service maintains its own CI/CD configuration.

This allows:

  • Independent builds
  • Faster pipelines
  • Microservice-level deployment

Stage 3 — Post Deploy Cleanup

cleanup-merged-branches

Purpose:

Automatically delete remote branches that have already been merged.

The script checks merged branches using:

git branch -r –merged origin/main –merged origin/production

Branches that are already merged into main or production are deleted automatically.

Protected branches are excluded:

main
production
test
dev

Example deletion command:

git push origin –delete branch-name

This keeps the repository clean by removing obsolete branches.


Branch Rules

The pipeline runs only for specific branches.

For detection and service triggers:

dev production test

For branch cleanup:

main production

CI/CD Templates Documentation

Overview

The gitlab/templates directory contains reusable CI/CD pipeline templates used by different microservices in the cotrav-platform monorepo.

These templates standardize the build, test, validation, and deployment processes across all services.

The pipelines are executed using GitLab Runner and orchestrated through GitLab CI/CD.

Using templates ensures:

  • Consistent pipeline configuration
  • Reduced duplication
  • Easier maintenance
  • Scalable CI/CD architecture for microservices
Template Directory Structure

gitlab/templates
│
├── build-service.yml
├── test-service.yml
├── validate-service.yml
├── deploy-service.yml
└── service-pipeline.yml

Each file represents a stage or workflow used by service pipelines.

1. build-service.yml

Purpose: Builds the service and prepares it for deployment.

Typical tasks include:

  • Installing dependencies
  • Compiling TypeScript
  • Building application artifacts
  • Creating Docker images

Example responsibilities:

Install dependencies Build application Prepare build artifacts

This template ensures every microservice follows the same build process.


2. test-service.yml

Purpose: Runs automated tests for the service.

Typical tasks:

  • Unit tests
  • Integration tests
  • Code quality checks

Example flow:

Install dependencies
Run test suite
Generate test reports

Testing ensures code stability before deployment.


3. validate-service.yml

Purpose: Validates the service configuration and code structure.

Validation checks may include:

  • Linting
  • Configuration validation
  • Dependency checks
  • Code formatting

Example:



Run linter Validate environment configuration Check dependencies

This stage prevents invalid code from progressing through the pipeline.


4. deploy-service.yml

Purpose: Deploys the service to the target environment.

Deployment tasks may include:

  • Building Docker containers using Docker
  • Pushing images to container registry
  • Running containers on the server
  • Restarting services
  • Updating NGINX routing if required

Example deployment flow:

Build Docker image
Push image to registry
Run container on server
Verify deployment

5. service-pipeline.yml

Purpose: Defines the full pipeline for a microservice by combining the templates.

This file typically includes:

test
validate
build
deploy

Example pipeline order:

 test → build -> validate -> deploy

Each microservice includes this template to run its CI/CD pipeline.


How Services Use Templates

Each microservice pipeline includes these templates using GitLab include syntax.

Example:

include: - local: gitlab/templates/service-pipeline.yml

This allows all services to share the same CI/CD logic.

CI/CD Template Architecture
Monorepo Pipeline
│
▼
Detect Changed Services
│
▼
Trigger Service Pipeline
│
▼
Service Pipeline
│
├── Validate Service
├── Build Service
├── Test Service
└── Deploy Service

Benefits of Template-Based Pipelines

Standardized CI/CD

All services follow the same pipeline structure.

Easier Maintenance

Updating one template updates pipelines for all services.

Faster Development

Developers don't need to write new CI/CD pipelines.

Scalable Architecture

New microservices can be added easily without creating new pipelin



Shared Module Documentation

Overview

The shared/ directory contains reusable code, utilities, and deployment scripts that are used across multiple services in the cotrav-platform monorepo.

This directory helps maintain consistency across services and avoids duplication of common functionality.

Shared modules are used by backend services located in:

Cotrav_Services/services/

This structure supports a scalable monorepo architecture integrated with GitLab CI/CD pipelines.


Directory Structure

shared
│
├── common
│
└── scripts
└── deploy.sh

1. common Directory

Purpose: Contains reusable modules or utilities shared across multiple services.

Typical contents may include:

  • Common helper functions
  • Shared configuration utilities
  • Logging utilities
  • Error handling modules
  • Middleware helpers

Example usage inside services:

import { logger } from "../../shared/common/logger"

Benefits:

  • Reduces duplicated code
  • Ensures consistent functionality across services
  • Makes maintenance easier

2. scripts Directory

Purpose: Contains automation scripts used during deployment or CI/CD execution.

Example:

shared/scripts/deploy.sh

This script is typically executed by GitLab Runner during the deployment stage of GitLab CI/CD pipelines.


deploy.sh Script

Purpose: Automates deployment of services to the server.

Typical tasks performed:

  • Pull latest code or image
  • Stop existing containers
  • Build or pull new container images using Docker
  • Start new containers
  • Reload NGINX if required

Example deployment flow:

Pull latest code
↓
Build Docker Image
↓
Stop old container
↓
Start new container
↓
Verify deployment

How Shared Modules Are Used

Services import utilities from the shared directory.

Example:

Cotrav_Services/services/auth-service
Cotrav_Services/services/user-service
Cotrav_Services/services/booking-service

Each service can reuse modules from:

shared/common

Architecture Diagram

Shared modules sit between services and utilities.

shared/common
│
│
┌────────────┼────────────┐
│ │ │
auth-service user-service booking-service
│ │ │
└────────────┴────────────┘

Benefits of Shared Modules

Code Reusability

Common logic is written once and reused everywhere.

Consistency

All services follow the same logging, error handling, and configuration standards.

Easier Maintenance

Updating shared modules automatically improves all services.

Faster Development

Developers can reuse existing utilities instead of rewriting code.



Deployment Script Documentation

Overview

deploy.sh is the core deployment automation script used in the cotrav-platform CI/CD pipeline.

The script is executed by GitLab CI/CD runners to deploy backend services using Docker and Docker Compose.

It supports:

  • Automated container deployment
  • Blue-Green production deployments
  • Automatic rollback on failure
  • Environment configuration loading
  • Nginx traffic switching
  • Health checks before switching traffic

This script enables zero-downtime deployments.


Deployment Flow

GitLab Pipeline │ │ ▼ deploy-service.yml │ │ ▼ deploy.sh │ ├── Load Environment ├── Pull Docker Image ├── Deploy Container ├── Health Check ├── Blue/Green Switch └── Nginx Reload

Script Arguments

The script accepts the following parameters:

deploy.sh <service_name> <environment> <docker_image> <docker_tag> <port> [env_file] [internal_port] [compose_file]

Example

./deploy.sh auth-service prod cotrav/auth-service v1.2.3 6001

Supported Environments

The deployment script supports three environments:

EnvironmentDescription
devDevelopment environment
testQA / testing environment
prodProduction environment with Blue-Green deployment

Directory Structure Used

The script automatically manages directories inside:

/opt/cotrav/CICD_Pipeline

Structure:

CICD_Pipeline │ ├── services │ └── auth-service │ └── docker-compose.yml │ ├── environments │ └── auth-service │ └── prod │ └── .env │ └── logs └── auth-service └── prod


These directories store:

  • service configuration
  • environment variables
  • container logs

Blue-Green Deployment

Production deployments use a Blue-Green strategy.

Two containers run on different ports:

ColorPort
Blue6001
Green6002

Deployment process:


Current live → BLUE

New deployment
│
▼
Deploy GREEN container
│
▼
Run health checks
│
▼
Switch Nginx traffic
│
▼
GREEN becomes active

This ensures zero downtime.


Environment Variable Loading

Environment variables are loaded from:

/opt/cotrav/CICD_Pipeline/environments/<service>/<env>/.env

The script uses a **safe loader**  function:

safe_source()

Features:

  • ignores comments
  • ignores invalid keys
  • prevents arbitrary bash execution
  • supports quoted values

Docker Deployment Process

Deployment steps:

1️⃣ Pull Docker image

docker pull image:tag

2️⃣ Deploy container

docker compose up -d

3️⃣ Force recreate container

–force-recreate

4️⃣ Remove unused containers

–remove-orphans

Health Check System

Before traffic switching, the script performs three layers of health checks.

Layer 1 – Container State

Checks if container is running.

docker inspect

Layer 2 – Port Listening

Checks if service port is open.

ss -tln

Layer 3 – HTTP Health Endpoint

If available, checks:

/health

Example:

http://localhost:6001/health

Deployment proceeds only if all checks pass.


Automatic Rollback

If deployment fails, the script automatically rolls back.

Rollback logic:

Deployment fails
│
▼
Previous image detected
│
▼
Restart container using previous image

Example:

docker compose up -d –force-recreate

If no previous image exists:

docker compose down

Nginx Traffic Switching

After a successful deployment in production, the script switches traffic using NGINX.

Example configuration:

upstream auth-service {
server localhost:6001;
# server localhost:6002;
}

After deployment:

upstream auth-service {
# server localhost:6001;
server localhost:6002;
}

Then Nginx reloads:

nginx -s reload

Logging System

The script automatically creates structured logs:

logs/auth-service/prod
│
├── info
├── error
└── combined

Permissions are configured automatically during deployment.


Resource Limits

The script automatically configures container resource limits.

Production

MEM_LIMIT = 2G
CPU_LIMIT = 1.0
RESTART_POLICY = always

Dev / Test

MEM_LIMIT = 512M CPU_LIMIT = 0.5 RESTART_POLICY = on-failure

Network Configuration

All containers run in a shared Docker network:

cotrav-network

This allows services to communicate internally.


Deployment Success Output

If deployment succeeds:

DEPLOYMENT SUCCESSFUL
Service : auth-service
Env : prod
Image : cotrav/auth-service:v1.2.3
Port : 6002
Color : green

Key Benefits

Zero Downtime Deployments

Blue-Green deployment prevents service interruption.

Automatic Rollback

System recovers automatically if deployment fails.

Environment Isolation

Separate configuration for dev, test, and production.

Secure Environment Loading

Prevents execution of unsafe shell commands.

Automated Infrastructure

Deployment, logging, networking, and Nginx switching are fully automated.

Cotrav_Services Documentation

Overview

Cotrav_Services is the backend monorepo that contains all microservices, shared packages, and deployment scripts for the Cotrav platform.

The repository is structured using a monorepo architecture powered by pnpm workspaces and Turborepo.

This setup allows multiple services and packages to share code efficiently while maintaining independent deployments.


Directory Structure

Cotrav_Services │ ├── packages │ ├── logger │ ├── errors │ ├── middlewares │ └── common │ ├── services │ ├── auth-service │ ├── user-service │ ├── booking-service │ └── other microservices │ ├── scripts │ └── deploy.sh │ ├── pnpm-workspace.yaml ├── turbo.json ├── tsconfig.json ├── package.json └── LICENSE

1. packages Directory

Purpose

The packages folder contains shared libraries used across all backend services.

These packages ensure:

  • Code reusability
  • Consistent architecture
  • Reduced duplication

Example Packages

PackagePurpose
loggerCentralized logging system
errorsCustom error handling
middlewaresExpress middleware utilities
commonShared helper functions

Example usage inside services:

import logger from "@cotrav/logger"

Benefits:

  • All services use the same logging system
  • Consistent error handling across APIs
  • Shared middleware architecture

2. services Directory

Purpose

The services folder contains individual microservices.

Each service is:

  • independently deployable
  • containerized using Docker
  • deployed through GitLab CI/CD

Example services:

services
├── auth-service
├── user-service
├── booking-service
└── gateway-service

Each service typically contains:

auth-service
│
├── src
├── Dockerfile
├── package.json
├── tsconfig.json
└── .gitlab-ci.yml

Responsibilities of services:

  • handle business logic
  • expose REST APIs
  • communicate with databases
  • integrate with other services

3. scripts Directory

Purpose

The scripts directory contains deployment automation tools.

Example:

scripts
└── deploy.sh

The deploy.sh script is used in production deployments.

It performs:

  • Docker container deployment
  • Blue-Green deployment
  • Health checks
  • Rollback on failure
  • Traffic switching via NGINX

4. pnpm-workspace.yaml

This file defines the monorepo workspace configuration.

Example:

packages:
- "packages/*"
- "services/*"

This allows all services and packages to share dependencies using **pnpm**.

Benefits:

  • Faster installations
  • Disk space optimization
  • Dependency deduplication

5. turbo.json

turbo.json configures build pipelines using Turborepo.

Example pipeline tasks:

build
lint
test
dev

Turborepo provides:

  • parallel builds
  • caching
  • dependency graph optimization

6. TypeScript Configuration

The project uses shared TypeScript configuration.

Files:

tsconfig.base.json
tsconfig.json

Purpose:

  • shared compiler settings
  • consistent TypeScript rules
  • simplified service setup

Monorepo Architecture Diagram

Cotrav Platform Monorepo
│
│
Cotrav_Services
│
┌─────────────────┴─────────────────┐
│ │
packages services
│ │
logger / errors / middlewares auth-service
common utilities booking-service
user-service
gateway

Backend Infrastructure

Internet
│
▼
Nginx Gateway
│
▼
Microservices (Docker Containers)
│
├── auth-service
├── Hotel-service
└── flight-service

Advantages of This Architecture

Scalable Microservices

Services can be deployed independently.

Code Reusability

Shared packages prevent code duplication.

Faster Builds

Caching and parallel builds using Turborepo.

Efficient Dependency Management

Handled by pnpm.

Production-Ready Deployment

Automated CI/CD pipeline with GitLab CI/CD.

Impliments to exmaple CICD ROOT SERVICES CICD

Auth Service Documentation Exmaple..

Overview

auth-service is a microservice responsible for authentication and authorization within the Cotrav platform.

It is built using Node.js and containerized using Docker. The service is deployed through GitLab CI/CD pipelines.

Responsibilities of this service include:

  • User authentication
  • Token generation
  • Authorization middleware
  • Secure API access

Service Folder Structure


auth-service │ ├── src │ ├── controllers │ ├── routes │ ├── services │ ├── middlewares │ └── index.ts │ ├── Dockerfile ├── docker-compose.yml ├── nginx.conf ├── .gitlab-ci.yml ├── package.json ├── pnpm-lock.yaml └── tsconfig.json

Source Code (src)

The ''src''  directory contains the core application logic.

Typical structure:

src │ ├── controllers # API request handlers ├── routes # Express route definitions ├── services # Business logic layer ├── middlewares # Authentication & validation └── index.ts # Application entry point

Responsibilities

ComponentDescription
ControllersHandle incoming HTTP requests
RoutesDefine API endpoints
ServicesBusiness logic implementation
MiddlewaresAuthentication and request validation

Dockerfile

The Dockerfile defines how the service container is built using Docker.

Responsibilities:

  • Install dependencies
  • Build TypeScript code
  • Start the Node.js application

Example workflow:

Build Image ↓ Install Dependencies ↓ Compile TypeScript ↓ Run Service

docker-compose.yml

''docker-compose.yml''  is used for **local development and testing**.

It allows developers to quickly run the service with required dependencies.

Example usage:

docker-compose up -d

Benefits:

*

Easy local setup
Container orchestration for development
Environment configuration

nginx.conf

The nginx.conf file configures NGINX to route traffic to the service container.

Responsibilities:

  • Reverse proxy configuration
  • Load balancing
  • Port routing
  • Security headers

Traffic flow:



Client Request │ ▼ Nginx Reverse Proxy │ ▼ Auth Service Container


.gitlab-ci.yml

The .gitlab-ci.yml file defines the CI/CD pipeline for the auth-service using GitLab CI/CD.

Pipeline stages may include:

  1. Validate service
  2. Run tests
  3. Build Docker image
  4. Push image to registry
  5. Deploy service

Example pipeline flow:



Code Push │ ▼ GitLab Pipeline │ ├── Install Dependencies ├── Run Tests ├── Build Docker Image ├── Push Image ▼ Deployment


Deployment Architecture

The service runs inside Docker containers and is exposed through NGINX.



Internet │ ▼ Nginx Gateway │ ▼ Auth Service Container


Configuration Files

FilePurpose
package.jsonNode.js project configuration
pnpm-lock.yamlDependency lock file
tsconfig.jsonTypeScript compiler configuration

Dependencies are managed using pnpm.


Integration with Monorepo

auth-service is part of the backend monorepo managed by Turborepo.

Shared packages from the monorepo can be used inside the service:

Example:



import logger from “@cotrav/logger”


Advantages of This Setup

  • Microservice architecture
  • Containerized deployment
  • Automated CI/CD pipeline
  • Scalable infrastructure
  • Shared code through monorepo packages
gitlab_ci_cd_documentation/cotrav-platform.1774867459.txt.gz · Last modified: by raviraj