Stackdriver Kubernetes Monitoring

Scenario Context - 1

We want to monitor not only the platform our services are running on, but also our services themselves. Plus setting up alerts when certain errors or spike happens.

Tutorial to Follow

Google Cloud’s default Stackdriver K8s Monitoring

Steps to do:

Scenario Context - 2

We want to automate the alert policy creation. Right now all the alerting policy in Stackdriver were manually created. We want to apply the policies to new environment programmatically next time. Therefore we are considerring some scripts to do the work.

Tutorial to Follow

GCP supports Managing Alerting Policies by API Managing Notification Channels by API

Steps to do:

  • gcloud CLI?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

1. Creating policies
```jshelllanguage
    gcloud alpha monitoring policies create --policy-from-file="rising-cpu-usage.yaml"
```

2. Retrieving policies
```jshelllanguage
    gcloud alpha monitoring policies list
```

3. Deleting policies
```jshelllanguage
    gcloud alpha monitoring policies delete projects/a-gcp-project/alertPolicies/12669073143329903307
```

4. Modifying policies
```jshelllanguage
    # Disable policy
    gcloud alpha monitoring policies update projects/a-gcp-project/alertPolicies/12669073143329903307 --no-enabled
    
    # Update policy with notification channel
    gcloud alpha monitoring policies update projects/a-gcp-project/alertPolicies/12669073143329903307 \
    --add-notification-channels="projects/a-gcp-project/notificationChannels/1355376463305411567"
    
    # Update policy documents
    gcloud alpha monitoring policies update projects/a-gcp-project/alertPolicies/12669073143329903307 \
    --documentation-format="text/markdown" \
    --documentation-from-file="cpu-usage-doc.md"
```

Issues:

Reference:

comments powered by Disqus