Tuesday, January 14, 2020

Monitor Compute instance in Oracle Cloud using metrics

The purpose of this particular blog is to demonstrate on how to setup and receive notifications via email when the compute instance goes down.


My running compute instance:-




And we have metrics associated with it also such as memory utilization, CPU utilization etc.




I have the tenancy admin rights enabled for this. If not, then we need to assign the below policies to the group which belongs to the user.

Allow group UserGroup1 to read metrics in tenancy
Allow group UserGroup1 to manage alarms in tenancy
Allow group UserGroup1 to manage ons-topics in tenancy
Allow group UserGroup1 to manage ons-subscriptions in tenancy


Once it is done, create the notification subscription





In the notifications page, create a topic










                     Now we will create a subscription using the topic which was created earlier.


                 



           


          


At this stage, we should be getting en email to confirm the subscription. Once the subscription is done, the color will change to green

               


Next, we will be creating an alarm definition













This will now trigger an email when the metrics will not be available for more than 2 minutes. At this moment, we will stop the compute instance.

After we waited for more than 2 minutes, we should see the alarm status as below





And the email snippet



Once the instance comes up and running, the alarm status will be in Ok status again.




Hope this will help someone. Happy learning







Monday, January 13, 2020

Ansible in OCI

Ansible is  an orchestration tool which helps in provisioning/Configuration of infrastructure resources. Also it doesn't require any agents to be installed on the targets. Ansible uses ssh to connect to the host and execute command via the shell.

Okay, lets come down to the point.


How to install ansible

Create a directory ansible.
[opc@cloud ~]$ cd ansible/
[opc@cloud ansible]$ sudo yum install -y ansible


Once ansible is installed, we can check the version as

[opc@cloud ansible]$ ansible --version
ansible 2.7.9
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/opc/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Aug  7 2019, 08:19:52) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39.0.1)]

Install the ansible OCI module:-
[opc@cloud ansible]$ git clone https://github.com/oracle/oci-ansible-modules.git
Cloning into 'oci-ansible-modules'...
remote: Enumerating objects: 147, done.
remote: Counting objects: 100% (147/147), done.
remote: Compressing objects: 100% (109/109), done.
remote: Total 3221 (delta 83), reused 64 (delta 35), pack-reused 3074
Receiving objects: 100% (3221/3221), 9.94 MiB | 4.40 MiB/s, done.
Resolving deltas: 100% (2581/2581), done.
[opc@cloud ansible]$



Once these are installed, please make sure /home/opc/.oci file and crosscheck the details. When we run ansible, it is going to get the details from this file.

Test:-

Using OCI CLI, list the compartment ids in the tenancy.


[opc@cloud list_objects]$  oci compute instance list --compartment-id ocid1.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXirdc7bfecxavf2wu7q
WARNING: Python 2 support is ending on December 31, 2019. Future versions of OCI CLI after January 2020, will not be compatible with Python 2. To avoid interruption at that time, please install OCI CLI in a compatible Python 3 environment.
{
  "data": [
    {
      "agent-config": {
        "is-monitoring-disabled": false
      },
      "availability-domain": "yjnv:AP-SYDNEY-1-AD-1",
      "compartment-id": "ocid1.tenancy.oc1..XXXXXXXXXXXXXXXXXXXXXXXkkmredbjrirdc7bfecxavf2wu7q",
      "dedicated-vm-host-id": null,
      "defined-tags": {},
      "display-name": "cloud",
      "extended-metadata": {},
      "fault-domain": "FAULT-DOMAIN-1",
      "freeform-tags": {},
      "id": "ocid1.instance.XXXXXXXXXXXXXXXXXXXXXXXe7gpqcuexcxpa4dciogwvxgiwwufxmv7aqdnxb6ajhsuaflqhq",
      "image-id": "ocid1.image.ocXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXqmmehrprg6gdjxs4on5lpzwiv64przksa",
      "ipxe-script": null,
      "launch-mode": "PARAVIRTUALIZED",
      "launch-options": {
        "boot-volume-type": "PARAVIRTUALIZED",
        "firmware": "UEFI_64",
        "is-consistent-volume-naming-enabled": true,
        "is-pv-encryption-in-transit-enabled": true,
        "network-type": "PARAVIRTUALIZED",
        "remote-data-volume-type": "PARAVIRTUALIZED"
      },
      "lifecycle-state": "RUNNING",
      "metadata": {
        "ssh_authorized_keys": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAvh9K2yIdfa3XtbmK3hrE2YmIx7dijhP8pLGrIkM2cuevLe+GCE4wc9mzpsDXxobLGTtIvjTVqbj+ZaznzLxSKJtUD8AMRQW1yajM+55YTC9H8XUO4WloXUDOavhrpyrnl4OV69t1raZ7W9tJqtsxdFElNutq1QKP39X1HoUVCVqXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXQK3R+IjjzIXoODWXNdh48GFYkLK1K7ULvbtS2MPQ== rsa-key-20191215"
      },
      "region": "ap-sydney-1",
      "shape": "VM.Standard.E2.1.Micro",
      "source-details": {
        "boot-volume-size-in-gbs": null,
        "image-id": "ocid1.image.oc1..aaaaaXXXXXXXXXXXXXXXXXXXehrprg6gdjxs4on5lpzwiv64przksa",
        "kms-key-id": null,
        "source-type": "image"
      },
      "system-tags": {
        "orcl-cloud": {
          "free-tier-retained": "true"
        }
      },
      "time-created": "2019-12-15T18:07:45.645000+00:00",
      "time-maintenance-reboot-due": null
    }
  ]
}




Now we will verify if ansible is returning the same value or not.

Create a ansible playbook list_buckets.yml

---
- name : List summary of existing buckets in OCI object storage
  connection: local
  hosts: localhost
  tasks:
    - name: List bucket facts
      oci_bucket_facts:
         namespace_name: 'sd4XXXX'
         compartment_id: 'ocid1.tXXXXXXXXXXXmy5ox5j7o7w64v44kjpnjkkmredbjrirdc7bfecxavf2wu7q'
      register: result
    - name: Dump result
      debug:
        msg: '{{result}}'



----------------------



Please note:namespace name will be retirved from the tenancy page details.

[opc@cloud ~]$ ansible-playbook list_buckets.yml
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [List summary of existing buckets in OCI object storage] *******************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************
ok: [localhost]

TASK [List bucket facts] ********************************************************************************************************************************************************************
ok: [localhost]

TASK [Dump result] **************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": {
        "buckets": [
            {
                "compartment_id": "ocXXXXXXXXXXXXXX5j7o7w64v44kjpnjkkmredbjrirdc7bfecxavf2wu7q",
                "created_by": "ocid1.saml2idp.oc1..aaaaaaaawh2o7c7azqcmc23kzme7frbsmidnmses2ci7oif2jyjb2qegfvxq/XXXXXXXX@gmail.com",
                "defined_tags": null,
                "etag": "4c3cc183-10c4-4095-ba0c-c754cdc17b1c",
                "freeform_tags": null,
                "name": "bucket-20200105-2041",
                "namespace": "sd4XXXXXX",
                "time_created": "2020-01-05T15:11:44.180000+00:00"
            }
        ],
        "changed": false,
        "failed": false
    }
}

PLAY RECAP **********************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0


Thus we can see that the values returned through ansible and OCI cli are same. For more information please visit https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/ansiblegetstarted.htm#samples


Hope it helps someone. Happy learning.


Saturday, January 11, 2020

Terraform with Oracle Cloud

To automate our day to day administrative tasks, so far we have been acquainted with Shell scripts, python etc. But with the advent of Cloud, not only mundane manual tasks are getting automated, infact , a whole bunch of infrastructure are getting built. Here comes a infrastructure as a code from Hashicorp known as "Terraform". Now lets come down to my point on how to use terraform in Oracle Cloud.


We should have the following information ready:


tenancy ID
user ID--This should be for the non federated user.
compartment ID


API SSL Key:-
You will need an SSL key pair to enable Terraform to connect to the OCI API under your identity. Start by generating a key

user@computer$ openssl genrsa -out oci_api_key.pem 2048
set file access to owner only read and write

user@computer$ chmod 600 oci_api_key.pem
generate the public half of the key pair

openssl rsa -pubout -in oci_api_key.pem -out oci_api_key_public.pem

[opc@cloud .oci]$ ls -lrt
total 16
-rw-------. 1 opc opc  299 Dec 16 10:12 config
-rw-------. 1 opc opc   48 Dec 17 16:41 oci_api_key_fingerprint
-rw-------. 1 opc opc 1679 Dec 19 10:56 oci_api_key.pem
-rw-------. 1 opc opc  451 Dec 19 10:57 oci_api_key_public.pem


The public key needs to be added to your user account in the OCI console. Open the account page for your user and select the "Add Public Key" button. Copy and paste the contents of the oci_api_key_public.pem file in the box of the "Add Public Key" dialog as shown below.






Terraform Executable
The Terraform executable is available at terraform.io Available for Linux, Mac Windows etc. pick the binary that matches your PC. The download contains a single executable file. Unzip it to a directory that is on your PATH for example /home/me/bin. In order to update the executable when a new version is released simply download the new version, unzip and replace the executable.

Check the terraform version:-

 terraform -version


Terraform v0.12.17

Your version of Terraform is out of date! The latest version
is 0.12.19. You can update by downloading from https://www.terraform.io/downloads.html
[opc@cloud ~]$



Create two files now
env-vars and provider.tf


[opc@cloud terraformfinal]$ cat env-vars
export TF_VAR_tenancy_ocid=ocid1.XXXXXXXXXXXXXXXXXXXXXXXX
export TF_VAR_user_ocid=ocid1.XXXXXXXXXXXXXXXX
export TF_VAR_fingerprint=$(cat ~/.oci/oci_api_key_fingerprint)
export TF_VAR_private_key_path=~/.oci/oci_api_key.pem
export TF_VAR_ssh_public_key=$(cat ~/.ssh/id_rsa.pub)
export TF_VAR_ssh_private_key=$(cat ~/.ssh/id_rsa)
export TF_VAR_region=ap-XXXXXXXXXXX



[opc@cloud terraformfinal]$ cat provider.tf
variable "tenancy_ocid" {}
variable "user_ocid" {}
variable "fingerprint" {}
variable "private_key_path" {}
variable "ssh_public_key" {}
variable "region" {}

provider "oci" {
  tenancy_ocid         = "${var.tenancy_ocid}"
  user_ocid            = "${var.user_ocid}"
  fingerprint          = "${var.fingerprint}"
  private_key_path     = "${var.private_key_path}"
  region               = "${var.region}"
  disable_auto_retries = "true"
}
resource "oci_identity_compartment" "CreateCompartment" {
  compartment_id = "ocid1.XXXXXXXXXXXXXXXXXX"
  description    = "var.compartment_description"
  name           = "var.compartment_name"
}
[opc@cloud terraformfinal]$

Go to the website https://www.terraform.io/docs/providers/oci/r/core_vcn.html for getting the required codes for creating the components.


Now source the env-vars file
source env-vars


At this moment, we are now ready to run the terraform commands.

[opc@cloud terraformfinal]$ terraform init

Initializing the backend...

Initializing provider plugins...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.oci: version = "~> 3.51"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
[opc@cloud terraformfinal]$


[opc@cloud terraformfinal]$ terraform plan

Warning: Interpolation-only expressions are deprecated

  on provider.tf line 9, in provider "oci":
   9:   tenancy_ocid         = "${var.tenancy_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.


Warning: Interpolation-only expressions are deprecated

  on provider.tf line 10, in provider "oci":
  10:   user_ocid            = "${var.user_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 3 more similar warnings elsewhere)

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

oci_identity_compartment.CreateCompartment: Refreshing state... [id=ocid1XXXXXXXXXXXXXXXXXXXXXfngjhqchro7j4u76hwudxqfpsaohkqrro6ffrnkyzjny3fyysq]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # oci_identity_compartment.CreateCompartment will be created
  + resource "oci_identity_compartment" "CreateCompartment" {
      + compartment_id = "ocid1.XXXXXXXXXXXXXXXaaa566jmy5ox5j7o7w64v44kjpnjkkmredbjrirdc7bfecxavf2wu7q"
      + defined_tags   = (known after apply)
      + description    = "var.compartment_description"
      + freeform_tags  = (known after apply)
      + id             = (known after apply)
      + inactive_state = (known after apply)
      + is_accessible  = (known after apply)
      + name           = "var.compartment_name"
      + state          = (known after apply)
      + time_created   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Warning: Interpolation-only expressions are deprecated

  on provider.tf line 9, in provider "oci":
   9:   tenancy_ocid         = "${var.tenancy_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.


Warning: Interpolation-only expressions are deprecated

  on provider.tf line 10, in provider "oci":
  10:   user_ocid            = "${var.user_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 3 more similar warnings elsewhere)


------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

[opc@cloud terraformfinal]$

[opc@cloud terraformfinal]$ terraform apply

Warning: Interpolation-only expressions are deprecated

  on provider.tf line 9, in provider "oci":
   9:   tenancy_ocid         = "${var.tenancy_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.


Warning: Interpolation-only expressions are deprecated

  on provider.tf line 10, in provider "oci":
  10:   user_ocid            = "${var.user_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 3 more similar warnings elsewhere)

oci_identity_compartment.CreateCompartment: Refreshing state... [id=ocid1.compartment.oc1..aaaaaaaacnfngjhqchro7j4u76hwudxqfpsaohkqrro6ffrnkyzjny3fyysq]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # oci_identity_compartment.CreateCompartment will be created
  + resource "oci_identity_compartment" "CreateCompartment" {
      + compartment_id = "ocid1.XXXXXXXXXXXXXXXXXX7w64v44kjpnjkkmredbjrirdc7bfecxavf2wu7q"
      + defined_tags   = (known after apply)
      + description    = "var.compartment_description"
      + freeform_tags  = (known after apply)
      + id             = (known after apply)
      + inactive_state = (known after apply)
      + is_accessible  = (known after apply)
      + name           = "var.compartment_name"
      + state          = (known after apply)
      + time_created   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.


Warning: Interpolation-only expressions are deprecated

  on provider.tf line 9, in provider "oci":
   9:   tenancy_ocid         = "${var.tenancy_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.


Warning: Interpolation-only expressions are deprecated

  on provider.tf line 10, in provider "oci":
  10:   user_ocid            = "${var.user_ocid}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 3 more similar warnings elsewhere)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

oci_identity_compartment.CreateCompartment: Creating...
oci_identity_compartment.CreateCompartment: Creation complete after 0s [id=ocid1.XXXXXXXXXXXXX.7naistwkyvrh6q4bzbht567n2f4xy53jatszucuhdek355zwfkq]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
[opc@cloud terraformfinal]$





Now login to the console and verify.





The resource can be destroyed by

terraform destroy



Hope, this will help someone. Happy learning.

















Installing OCI-CLI in Oracle Cloud

In this blog post, i will describe installation and configuration of command line interface in Oracle Cloud. Before we start, we should have the below details ready.

1. OCI Users
2. SSH Keypair used for signing API requests, with the public key being uploaded to the Oracle OCI.
3. Python

We should have python version 3 and above else we may get the following message:-

WARNING: Python 2 support is ending on December 31, 2019. Future versions of OCI CLI after January 2020, will not be compatible with Python 2. To avoid interruption at that time, please install OCI CLI in a compatible Python 3 environment.


Installing Python:-

Using root user

yum install gcc libffi-devel python-devel openssl-devel

yum install python36

ln -s /usr/bin/python36 /usr/bin/python3


[root@cloud opc]# python3 --version
Python 3.6.8



Installing CLI:-

bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)"



Configuring CLI:-


We should have the below details ready:-


Tenancy OCID
Compartment OCID
User OCID
SSH Keys





[opc@cloud ~]$ oci setup config

Enter a location for your config [/home/opc/.oci/config]:
Enter a user OCID: ocid1.user.oc1..aaaaaaaaxvnXXXXXXXXXXXXXXXXXXXXXXX
Enter a tenancy OCID: ocid1.tenancy.oc1..XXXXXXXXXXXXXXXXXXXXXX
Enter a region (e.g. ap-mumbai-1, ap-seoul-1, ap-sydney-1, ap-tokyo-1, ca-toronto-1, eu-frankfurt-1, eu-zurich-1, sa-saopaulo-1, uk-london-1, us-ashburn-1, us-gov-ashburn-1, us-gov-chicago-1, us-gov-phoenix-1, us-langley-1, us-luke-1, us-phoenix-1): XXXXXX
Do you want to generate a new RSA key pair? (If you decline you will be asked to supply the path to an existing key.) [Y/n]: y
Enter a directory for your keys to be created [/home/opc/.oci]:
Enter a name for your key [oci_api_key]:
Public key written to: /home/opc/.oci/oci_api_key_public.pem
Enter a passphrase for your private key (empty for no passphrase):
Private key written to: /home/opc/.oci/oci_api_key.pem
Fingerprint: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Config written to /home/opc/.oci/config


copy the public key to the public key content to the API keys section of the OCI user


Once we upload the key, it will generate a fingerprint which should match the fingerprint generated during CLI setup section.



Now verify if CLI setup has been done properly or not.

[opc@cloud .oci]$ oci iam availability-domain list







[opc@cloud ~]$ oci network vcn list --compartment-id "Compartment OCID"




Now, we can use CLI to automate our administrative tasks. Hope, it helps someone

Happy learning cloud....................









Friday, January 10, 2020

How to Get the usage report in Oracle Cloud OCI

Usage reports are CSV files generated daily that show usage data for each resource in your tenancy. The CSV files are stored in an object storage bucket that is accessible using a cross-tenancy policy. 

The reports are stored in the Oracle owned object storage bucket. The reports are retained for one year, however it can be modified later on.

Normally, we cannot view the usage report directly. Our account should have the proper IAM policies attached to it.

The two policies are :-


define tenancy usage-report as ocid1.tenancy.oc1..aaaaaaaaned4fkpkisbwjlr56u7cj63lf3wffbilvqknstgtvzub7vhqkggq

endorse group "group name"to read objects in tenancy usage-report

I have created the usagepolicy as show below:-




                      There are two policy statements included.

Once the policy is created, we will be able to view the usage report which will be available under Account Management>>Usage report section




Download the reports in your PC and have a look into it. It contains one row per each Oracle Cloud Infrastructure resource (such as instance, Object Storage bucket, VNIC) per hour along with consumption information, metadata, and tags. Usage reports generally contain 24 hours of usage data, although occasionally a usage report may contain late-arriving data that is older than 24 hours.

Hope this helps someone. Happy learning Cloud......................................