wiki

User Tools

Site Tools

Trace:
gitlab_ci_cd_documentation:cotrav-platform

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
gitlab_ci_cd_documentation:cotrav-platform [2026/03/30 10:17] – [Layer 2 – Port Listening] ravirajgitlab_ci_cd_documentation:cotrav-platform [2026/03/31 05:54] (current) – [Stage 1 — Test] raviraj
Line 869: Line 869:
  
 ---- ----
- 
  
 ==== Layer 3 – HTTP Health Endpoint ==== ==== Layer 3 – HTTP Health Endpoint ====
Line 995: Line 994:
  
 ---- ----
- 
  
 ====== Network Configuration ====== ====== Network Configuration ======
Line 1063: Line 1061:
  
 <file> <file>
- 
 Cotrav_Services │ ├── packages │ ├── logger │ ├── errors │ ├── middlewares │ └── common │ ├── services │ ├── auth-service │ ├── user-service │ ├── booking-service │ └── other microservices │ ├── scripts │ └── deploy.sh │ ├── pnpm-workspace.yaml ├── turbo.json ├── tsconfig.json ├── package.json └── LICENSE Cotrav_Services │ ├── packages │ ├── logger │ ├── errors │ ├── middlewares │ └── common │ ├── services │ ├── auth-service │ ├── user-service │ ├── booking-service │ └── other microservices │ ├── scripts │ └── deploy.sh │ ├── pnpm-workspace.yaml ├── turbo.json ├── tsconfig.json ├── package.json └── LICENSE
  
Line 1069: Line 1066:
  
 ---- ----
- 
  
 ====== 1. packages Directory ====== ====== 1. packages Directory ======
Line 1105: Line 1101:
  
 ---- ----
- 
  
 ====== 2. services Directory ====== ====== 2. services Directory ======
Line 1111: Line 1106:
 ===== Purpose ===== ===== Purpose =====
  
-The ''services'' folder contains individual **microservices**.+The ''services''  folder contains individual **microservices**.
  
 Each service is: Each service is:
Line 1151: Line 1146:
  
 ---- ----
- 
  
 ====== 3. scripts Directory ====== ====== 3. scripts Directory ======
Line 1157: Line 1151:
 ===== Purpose ===== ===== Purpose =====
  
-The ''scripts'' directory contains deployment automation tools.+The ''scripts''  directory contains deployment automation tools.
  
 Example: Example:
Line 1167: Line 1161:
 </file> </file>
  
-The ''deploy.sh'' script is used in production deployments.+The ''deploy.sh''  script is used in production deployments.
  
 It performs: It performs:
Line 1178: Line 1172:
  
 ---- ----
- 
  
 ====== 4. pnpm-workspace.yaml ====== ====== 4. pnpm-workspace.yaml ======
Line 1202: Line 1195:
  
 ---- ----
- 
  
 ====== 5. turbo.json ====== ====== 5. turbo.json ======
  
-''turbo.json'' configures **build pipelines** using **Turborepo**.+''turbo.json''  configures **build pipelines**  using **Turborepo**.
  
 Example pipeline tasks: Example pipeline tasks:
Line 1225: Line 1217:
  
 ---- ----
- 
  
 ====== 6. TypeScript Configuration ====== ====== 6. TypeScript Configuration ======
Line 1246: Line 1237:
  
 ---- ----
- 
  
 ====== Monorepo Architecture Diagram ====== ====== Monorepo Architecture Diagram ======
Line 1268: Line 1258:
  
 ---- ----
- 
  
 ====== Backend Infrastructure ====== ====== Backend Infrastructure ======
Line 1282: Line 1271:
 │ │
 ├── auth-service ├── auth-service
-├── booking-service +├── Hotel-service 
-└── user-service+└── flight-service
  
 </code> </code>
  
 ---- ----
- 
  
 ====== Advantages of This Architecture ====== ====== Advantages of This Architecture ======
Line 1311: Line 1299:
  
 Automated CI/CD pipeline with **GitLab CI/CD**. Automated CI/CD pipeline with **GitLab CI/CD**.
 +
 +=====   =====
 +
 +===== Impliments to exmaple CICD ROOT SERVICES CICD =====
 +
 +
 +====== Auth Service Documentation Exmaple.. ======
 +
 +
 +===== Overview =====
 +
 +''auth-service''  is a microservice responsible for authentication and authorization within the Cotrav platform.
 +
 +It is built using **Node.js**  and containerized using **Docker**.  The service is deployed through **GitLab CI/CD**  pipelines.
 +
 +Responsibilities of this service include:
 +
 +  * User authentication
 +  * Token generation
 +  * Authorization middleware
 +  * Secure API access
 +
 +----
 +
 +====== Service Folder Structure ======
 +
 +<code>
 +
 +
 +</code>
 +
 +<code>
 +
 +auth-service │ ├── src │ ├── controllers │ ├── routes │ ├── services │ ├── middlewares │ └── index.ts │ ├── Dockerfile ├── docker-compose.yml ├── nginx.conf ├── .gitlab-ci.yml ├── package.json ├── pnpm-lock.yaml └── tsconfig.json
 +
 +</code>
 +
 +----
 +
 +
 +====== Source Code (src) ======
 +
 +<file>
 +The ''src''  directory contains the core application logic.
 +
 +Typical structure:
 +
 +src │ ├── controllers # API request handlers ├── routes # Express route definitions ├── services # Business logic layer ├── middlewares # Authentication & validation └── index.ts # Application entry point
 +
 +</file>
 +
 +
 +==== Responsibilities ====
 +
 +^Component^Description|
 +|Controllers|Handle incoming HTTP requests|
 +|Routes|Define API endpoints|
 +|Services|Business logic implementation|
 +|Middlewares|Authentication and request validation|
 +
 +----
 +
 +====== Dockerfile ======
 +
 +The ''Dockerfile'' defines how the service container is built using **Docker**.
 +
 +Responsibilities:
 +
 +  * Install dependencies
 +  * Build TypeScript code
 +  * Start the Node.js application
 +
 +Example workflow:
 +
 +<code>
 +Build Image ↓ Install Dependencies ↓ Compile TypeScript ↓ Run Service
 +
 +</code>
 +
 +----
 +
 +
 +====== docker-compose.yml ======
 +
 +<file>
 +''docker-compose.yml''  is used for **local development and testing**.
 +
 +It allows developers to quickly run the service with required dependencies.
 +
 +Example usage:
 +
 +docker-compose up -d
 +
 +</file>
 +
 +Benefits:
 +
 +Container orchestration for development
 +
 +Easy local setup
 +
 +Environment configuration
 +
 +----
 +
 +
 +====== nginx.conf ======
 +
 +The ''nginx.conf'' file configures **NGINX** to route traffic to the service container.
 +
 +Responsibilities:
 +
 +  * Reverse proxy configuration
 +  * Load balancing
 +  * Port routing
 +  * Security headers
 +
 +Traffic flow:
 +
 +<code>
 +Client Request │ ▼ Nginx Reverse Proxy │ ▼ Auth Service Container
 +
 +</code>
 +
 +----
 +
 +
 +====== .gitlab-ci.yml ======
 +
 +The ''.gitlab-ci.yml'' file defines the **CI/CD pipeline** for the auth-service using **GitLab CI/CD**.
 +
 +Pipeline stages may include:
 +
 +  - Validate service
 +  - Run tests
 +  - Build Docker image
 +  - Push image to registry
 +  - Deploy service
 +
 +Example pipeline flow:
 +
 +<code>
 +Code Push │ ▼ GitLab Pipeline │ ├── Install Dependencies ├── Run Tests ├── Build Docker Image ├── Push Image ▼ Deployment
 +
 +</code>
 +==== Project: Cotrav Services – Auth Service ====
 +
 +===== 1. Overview =====
 +
 +This project implements a **CI/CD pipeline using GitLab CI**  to automate the process of:
 +
 +  * Testing
 +  * Building
 +  * Validating
 +  * Deploying
 +  * Cleaning Docker images
 +
 +The pipeline is designed for a **Monorepo Microservices Architecture**  where multiple services exist inside a single repository. The pipeline specifically handles deployment for the **Auth Service**.
 +
 +----
 +
 +====== 2. Pipeline Architecture ======
 +
 +The CI/CD pipeline contains the following stages:
 +
 +<code>Developer Push Code
 +│
 +▼
 +GitLab Repository
 +│
 +▼
 +GitLab Runner (docker-runner01)
 +│
 +▼
 +TEST → BUILD → VALIDATE → DEPLOY → CLEANUP
 +│
 +▼
 +Docker Image Build
 +│
 +▼
 +Push to GitLab Container Registry
 +│
 +▼
 +SSH Deployment to Server
 +
 +</code>
 +
 +----
 +
 +====== 3. Pipeline Stages ======
 +
 +===== Stage 1 — Test =====
 +
 +<code>
 +test-auth
 +
 +</code>
 +
 +This stage runs automated tests before building the service.
 +
 +Purpose:
 +
 +  * Verify code functionality
 +  * Prevent broken builds
 +  * Ensure stability before deployment
 +
 +The job extends a reusable template:
 +
 +<code>
 +extends: .base_test_job
 +
 +</code>
 +
 +The runner used:
 +
 +<code>
 +tags:
 +- docker-runner01
 +
 +</code>
 +
 +----
 +
 +
 +====== 4. Stage 2 — Build ======
 +
 +<code>build-auth
 +
 +</code>
 +
 +Purpose:
 +
 +  * Build the Auth service
 +  * Create Docker image
 +  * Push image to GitLab Container Registry
 +
 +Dependency:
 +
 +<code>needs: ["test-auth"]
 +
 +</code>
 +
 +Build only runs after **tests pass**.
 +
 +Docker image naming format:
 +
 +<code>auth-service-${branch}-V02-${commit}
 +
 +</code>
 +
 +Example:
 +
 +<code>auth-service-dev-V02-a23f9c
 +
 +</code>
 +
 +----
 +
 +====== 5. Stage 3 — Validate ======
 +
 +<code>validate-auth
 +
 +</code>
 +
 +Purpose:
 +
 +  * Validate Docker image
 +  * Ensure artifacts are correct
 +  * Check build integrity
 +
 +Dependency:
 +
 +<code>needs:
 +- build-auth
 +
 +</code>
 +
 +Artifacts from build are used for validation.
 +
 +----
 +
 +====== 6. Deployment Strategy ======
 +
 +Deployment is automated using **SSH + Docker**.
 +
 +Pipeline supports **3 environments**:
 +
 +^Environment^Branch^Port|
 +|Dev|dev|4001|
 +|Test|test|4002|
 +|Production|production|4003|
 +
 +----
 +
 +====== 7. Dev Deployment ======
 +
 +Job:
 +<code>
 +deploy-dev
 +
 +</code>
 +
 +Runs automatically when:
 +
 +<code>branch == dev
 +
 +</code>
 +
 +Configuration:
 +
 +<code>PORT=4001
 +INTERNAL_PORT=3000
 +
 +</code>
 +
 +This deploys the container to the development environment.
 +
 +----
 +
 +====== 8. Test Deployment ======
 +
 +Job:
 +
 +<code>deploy-test
 +
 +</code>
 +
 +Runs when:
 +
 +<code>branch == test
 +
 +</code>
 +
 +Deployment is **manual approval based**:
 +
 +<code>when: manual
 +
 +</code>
 +
 +Configuration:
 +
 +<code>PORT=4002
 +
 +</code>
 +
 +This allows QA or testing before production release.
 +
 +----
 +
 +====== 9. Production Deployment ======
 +
 +Job:
 +
 +<code>deploy-prod
 +
 +</code>
 +
 +Production deployment uses **Blue-Green Deployment Strategy**.
 +
 +Configuration:
 +
 +<code>ex - BLUE_PORT=4003
 +GREEN_PORT=4004
 +
 +</code>
 +
 +Benefits:
 +
 +  * Zero downtime deployment
 +  * Safe rollback
 +  * High availability
 +
 +Deployment is also **manual approval based**.
 +
 +----
 +
 +====== 10. Pipeline Rules ======
 +
 +Pipeline runs only if **relevant files change**.
 +
 +<code>changes:
 +- Cotrav_Services/packages/**/*
 +- Cotrav_Services/services/auth-service/**/*
 +- Cotrav_Services/pnpm-lock.yaml
 +- Cotrav_Services/package.json
 +
 +</code>
 +
 +Benefits:
 +
 +  * Faster pipeline
 +  * Avoid unnecessary builds
 +
 +----
 +
 +====== 11. Caching Strategy ======
 +
 +Pipeline uses **PNPM caching**  to speed up builds.
 +
 +<code>cache:
 +key: pnpm-cache-$CI_COMMIT_REF_SLUG
 +
 +</code>
 +
 +Cached directories:
 +
 +<code>Cotrav_Services/.pnpm-store
 +Cotrav_Services/node_modules
 +node_modules
 +
 +</code>
 +
 +Benefits:
 +
 +  * Faster dependency installation
 +  * Reduced CI runtime
 +
 +----
 +
 +====== 12. Docker Build Optimization ======
 +
 +<code>DOCKER_BUILDKIT: "1"
 +
 +</code>
 +
 +BuildKit improves:
 +
 +  * Docker build performance
 +  * Layer caching
 +  * Parallel builds
 +
 +----
 +
 +====== 13. Environment Variables ======
 +
 +Important pipeline variables:
 +
 +^Variable^Purpose|
 +|RUNNER_TAG|Runner to execute jobs|
 +|SERVICE_PATH|Path to service|
 +|DOCKER_IMAGE|Docker registry image|
 +|DOCKER_TAG|Unique build tag|
 +|REMOTE_BASE_PATH|Deployment directory|
 +|ENV_CONTENT|Encoded environment file|
 +
 +Environment file is stored securely in GitLab CI variables:
 +<code>
 +AUTH_SERVICE_ENV_BASE64  /   this add cicd varible
 +
 +</code>
 +
 +----
 +
 +====== 14. Cleanup Stage ======
 +
 +<code>cleanup-registry
 +
 +</code>
 +
 +Purpose:
 +
 +Remove old Docker images from the registry.
 +
 +Command used:
 +
 +<code>glab registry delete
 +
 +</code>
 +
 +Configuration:
 +
 +<code>--keep-n 5
 +
 +</code>
 +
 +Meaning:
 +
 +Only **latest 5 images are kept**.
 +
 +Benefits:
 +
 +  * Saves registry storage
 +  * Maintains clean image history
 +
 +----
 +
 +====== 15. GitLab Templates ======
 +
 +Pipeline uses reusable templates:
 +
 +<file>
 +
 +gitlab/templates/test-service.yml
 +gitlab/templates/build-service.yml
 +gitlab/templates/validate-service.yml
 +gitlab/templates/deploy-service.yml
 +
 +</file>
 +
 +Advantages:
 +
 +  * Reusable CI logic
 +  * Cleaner ''.gitlab-ci.yml''
 +  * Easy to scale for multiple services
 +
 +----
 +
 +====== 16. Monorepo Structure ======
 +
 +Example project structure:
 +
 +<file>
 +Cotrav_Services
 +│
 +├── packages
 +│ ├── logger
 +│ ├── errors
 +│ └── middlewares
 +│
 +├── services
 +│ └── auth-service
 +│
 +├── scripts
 +│
 +├── pnpm-workspace.yaml
 +├── turbo.json
 +└── package.json
 +
 +Tools used:
 +PNPM TurboRepo TypeScript
 +
 +</file>
 +
 +<file>
 +include:
 +  - local: gitlab/templates/test-service.yml
 +  - local: gitlab/templates/build-service.yml
 +  - local: gitlab/templates/validate-service.yml
 +  - local: gitlab/templates/deploy-service.yml
 +
 +variables:
 +  RUNNER_TAG: "docker-runner01"
 +  DOCKER_BUILDKIT: "1"
 +  MONOREPO_ROOT: "Cotrav_Services"
 +  SERVICE_PATH: "Cotrav_Services/services/auth-service"
 +  BASE_SERVICE_NAME: "auth-service"
 +  VERSION_NUMBER: "V02"
 +  DOCKER_IMAGE: "$CI_REGISTRY_IMAGE/auth-service"
 +  DOCKER_TAG: "auth-service-${CI_COMMIT_REF_SLUG}-V02-${CI_COMMIT_SHORT_SHA}"
 +  REMOTE_BASE_PATH: "/opt/cotrav/CICD_Pipeline"
 +  ENV_CONTENT: "$AUTH_SERVICE_ENV_BASE64"
 +
 +stages:
 +  - test
 +  - build
 +  - validate
 +  - deploy
 +  - cleanup
 +
 +cache:
 +  key: "pnpm-cache-$CI_COMMIT_REF_SLUG"
 +  paths:
 +    - "Cotrav_Services/.pnpm-store"
 +    - "Cotrav_Services/node_modules"
 +    - "node_modules"
 +  policy: pull-push
 +
 +.auth-service_rules: &auth_rules
 +  rules:
 +    - if: '$CI_COMMIT_BRANCH == "dev" || $CI_COMMIT_BRANCH == "test" || $CI_COMMIT_BRANCH == "production"'
 +      changes:
 +        - "Cotrav_Services/packages/**/*"
 +        - "Cotrav_Services/services/auth-service/**/*"
 +        - "Cotrav_Services/pnpm-lock.yaml"
 +        - "Cotrav_Services/package.json"
 +# ...
 +
 +# ─── TEST ────────────────────────────────────────────────
 +test-auth:
 +  extends: .base_test_job
 +  tags:
 +    - "$RUNNER_TAG"
 +  <<: *auth_rules
 +
 +# ─── BUILD ───────────────────────────────────────────────
 +build-auth:
 +  extends: .base_build_job
 +  needs: ["test-auth"]
 +  tags:
 +    - "$RUNNER_TAG"
 +  <<: *auth_rules
 +
 +# ─── VALIDATE ────────────────────────────────────────────
 +validate-auth:
 +  extends: .base_validate_job
 +  stage: validate
 +  needs:
 +    - job: build-auth
 +      artifacts: true
 +  tags:
 +    - "$RUNNER_TAG"
 +  <<: *auth_rules
 +
 +# ─── DEPLOY DEV ──────────────────────────────────────────
 +deploy-dev:
 +  extends: .base_deploy_job
 +  needs:
 +    - job: build-auth
 +      artifacts: true
 +    - job: validate-auth
 +      artifacts: true
 +  variables:
 +    SERVICE_NAME: "auth-service"
 +    TARGET_ENV: "dev"
 +    DEPLOY_SERVER_IP: "$DEV_IP"
 +    SSH_USER: "root"
 +    PORT: "4001"
 +    INTERNAL_PORT: "3000"
 +  environment:
 +    name: dev/auth-service
 +  rules:
 +    - if: '$CI_COMMIT_BRANCH == "dev"'
 +
 +# ─── DEPLOY TEST ─────────────────────────────────────────
 +deploy-test:
 +  extends: .base_deploy_job
 +  needs:
 +    - job: build-auth
 +      artifacts: true
 +    - job: validate-auth
 +      artifacts: true
 +  variables:
 +    SERVICE_NAME: "auth-service"
 +    TARGET_ENV: "test"
 +    DEPLOY_SERVER_IP: "$DEV_IP"
 +    SSH_USER: "root"
 +    PORT: "4002"
 +    INTERNAL_PORT: "3000"
 +  environment:
 +    name: test/auth-service
 +  rules:
 +    - if: '$CI_COMMIT_BRANCH == "test"'
 +      when: manual
 +
 +# ─── DEPLOY PROD ─────────────────────────────────────────
 +deploy-prod:
 +  extends: .base_deploy_job
 +  needs:
 +    - job: build-auth
 +      artifacts: true
 +    - job: validate-auth
 +      artifacts: true
 +  variables:
 +    SERVICE_NAME: "auth-service"
 +    TARGET_ENV: "prod"
 +    DEPLOY_SERVER_IP: "$DEV_IP"
 +    SSH_USER: "root"
 +    BLUE_PORT: "4003"
 +    GREEN_PORT: "4004"
 +    PORT: "4003"
 +    INTERNAL_PORT: "3000"
 +  environment:
 +    name: prod/auth-service
 +  rules:
 +    - if: '$CI_COMMIT_BRANCH == "production"'
 +      when: manual
 +
 +# ─── CLEANUP ─────────────────────────────────────
 +cleanup-registry:
 +  stage: cleanup
 +  image: registry.gitlab.com/gitlab-org/cli:latest
 +  script:
 +    - echo "Cleaning up old images for $BASE_SERVICE_NAME"
 +    - glab registry delete "$CI_PROJECT_PATH" --name "$BASE_SERVICE_NAME" --keep-n 5 --yes || true
 +  rules:
 +    - if: '$CI_COMMIT_BRANCH == "production"'
 +      when: on_success
 +  allow_failure: true
 +
 +</file>
 +
 +----
 +
 +
 +====== 17. Deployment Server ======
 +
 +Deployment server path:
 +
 +<code>/opt/cotrav/CICD_Pipeline
 +
 +</code>
 +
 +Deployment method:
 +
 +<code>GitLab Runner → SSH → Docker Container
 +
 +</code>
 +
 +----
 +
 +====== 18. Advantages of This Pipeline ======
 +
 +  * Automated testing
 +  * Automated Docker build
 +  * Monorepo support
 +  * Environment based deployment
 +  * Blue-Green production deployment
 +  * Docker registry cleanup
 +  * Faster builds using caching
 +
 +----
 +
 +====== 19. Technologies Used ======
 +
 +^Technology^Purpose|
 +|GitLab CI/CD|Pipeline automation|
 +|Docker|Containerization|
 +|PNPM|Dependency management|
 +|TurboRepo|Monorepo build system|
 +|TypeScript|Backend development|
 +|Nginx|Reverse proxy|
 +|SSH|Remote deployment|
 +
 +----
 +
 +
 +====== Deployment Architecture ======
 +
 +The service runs inside Docker containers and is exposed through **NGINX**.
 +<file>
 +Internet │ ▼ Nginx Gateway │ ▼ Auth Service Container
 +
 +</file>
 +
 +----
 +
 +
 +====== Configuration Files ======
 +
 +^File^Purpose|
 +|package.json|Node.js project configuration|
 +|pnpm-lock.yaml|Dependency lock file|
 +|tsconfig.json|TypeScript compiler configuration|
 +
 +Dependencies are managed using **pnpm**.
 +
 +----
 +
 +====== Integration with Monorepo ======
 +
 +''auth-service'' is part of the backend monorepo managed by **Turborepo**.
 +
 +Shared packages from the monorepo can be used inside the service:
 +
 +Example:
 +
 +<code>
 +import logger from "@cotrav/logger"
 +
 +</code>
 +
 +----
 +
 +
 +====== Advantages of This Setup ======
 +
 +  * Microservice architecture
 +  * Containerized deployment
 +  * Automated CI/CD pipeline
 +  * Scalable infrastructure
 +  * Shared code through monorepo packages
  
  
gitlab_ci_cd_documentation/cotrav-platform.1774865827.txt.gz · Last modified: by raviraj