Tracking Deployments With New Relic

While working with microservice architecture and manage distributed systems that includes many of bounded contexts, you need to monitor your services one by one and be sure about the systems are up all time.

In the life cycle of software development(SDLC) we have 6 steps to be done.

Such as planning, analysis, design, implementation, testing, deployment

After every feature development we run our tests in Qa environment and check quality & functionality & code coverage

We should track our system’s behaviour before deployment and after deployment for knowing deployment effects to avg response time, throughput, cpu, memory, error rate and database.

If any anomaly that occur in production environment due to deployment this metrics will totally helpful to find root cause of the problem.

Record deployments

New Relic allows you to track deployments so you can correlate deploy to your app’s performance. Tracking deployments creates deployment markers that appear in APM charts.

Tracking Deployments Options

1. Record a deployment with POST

2. Record a deployment with PowerShell

If you are using CI/CD to deliver apps by automation, I suggest you to add deployment marker via curl request in deployment stage. Let’s see how we can mark our deployments in CI/CD pipeline

# A curl request example given in New Relic Deployment Docs

In this curl request above, we see a few properties for identify our deployment. Let’s talk about how we can set up these parametrically;

$APP_ID : App ID that you get from new relic app which you want to track

$API_KEY : The key you created with USER type for deployment marking from API keys section

I will show the example steps on Gitlab CI/CD

Gitlab Environment Variables To Identify Deployment

changelog : Commit ID = $CI_COMMIT_SHORT_SHA

description : PIPELINE ID = $CI_PIPELINE_ID - QA

timestamp : $CI_JOB_STARTED_AT

You can also use gitlab user info of account to identify deployer

user : $GITLAB_USER_LOGIN

How To Use Gitlab Environment Variables

In gitlab yml file, we have some stages to automate deployment such as docker-build, unit test, deploy-qa, integration test, deploy-prod that can be change due to your deployment strategies

Let’s talk about deploying your new features into the qa environment. In this senario think like you are using kubectl to push your new docker image into your k8s cluster

For using curl request in this phase, you need to install curl via this command before you send the request so you should set this command in before_script step as below;

before_script: 
- apk add --update curl && rm -rf /var/cache/apk/*

After installing curl, you will be ready to send curl request to mark your deployment

After you do success deployment, you set your curl command in after_script step as below;

We used gitlab ci variables that given to us but we can also create our own environment variables at gitlab under the Settings/CI CD/Variables section.

So you can set NEW_RELIC_DEPLOYMENT_KEY_QA variable here and use from everywhere (deployment, ingress, service yaml, ci yml etc) in your project. I prefer to use this key from environment variable in gitlab ci yml so I added the key into NEW_RELIC_DEPLOYMENT_KEY_QA and wrote as ${NEW_RELIC_DEPLOYMENT_KEY_QA} in ci yml

And after this configurations, we will be able to see effects in New Relic Deployment Metrics UI

New Relic Deployment UI Dashboards

  1. Deployments Table

2. Overview of response time, throughput, cpu, memory

3. Change Log Details

4. Detailed Change Report Which Including Endpoint And Its A lot Of Metrics

References

https://docs.newrelic.com/docs/apm/new-relic-apm/maintenance/record-monitor-deployments/

Software Engineer @hepsiburada