Python In DevOps

Repository
Features
- Easy to learn
- Large ecosystem
- Flexible
Uses
- Web dev
- ML
- AI
- Data Science
- DevOps
- Web scraping
- Automation
Why Python as a DevOps engineer?
- As a DevOps engineer there are many tools you need to combine like CI/CD pipelines, infrastructure management, monitoring, etc
- Other tasks
  - Automatically update Jira ticket after Jenkins build ran
  - Trigger Jenkins jobs on some events
  - Send notifications to team members on specific events
  - Regular backups
  - Clean up old Docker images
- Lots of repetitive tasks
- You'd want to automate these tasks as much as possible
- You may need to create automation scripts and programs for your teams
- Tasks you can automate
  - System Health Checks
  - Managing Cron
  - CI/CD related tasks
  - Cleanup tasks
  - Data visualization
  - Custom Ansible Modules
  - Backup tasks
  - Monitoring tasks

Installation and Local setup

Python Interpreter
- Knows how to execute Python code
- Translates our program into machine readable binary code

Variables

Python is dynamically typed
Naming convention used is snake case

Functions

Scope
- A variable is only available from inside the region it is created
Global Scope: variables are available from within any scope
Local Scope: variables created inside a function
Casting
- Convert a value from one data type into another

Conditionals

type() can be used to check variable types

Error handling

def validate_and_execute():
    try:
        user_input_number = int(num_of_days_element)

        # we want to do conversion only for positive integers
        if user_input_number > 0:
            calculated_value = days_to_units(user_input_number)
            print(calculated_value)
        elif user_input_number == 0:
            print("you entered a 0, please enter a valid positive number")
        else:
            print("you entered a negative number, no conversion for you!")
    except ValueError:
        print("your input is not a valid number. Don't ruin my program!")

Can leave except without warning type but IDE will warn you

Sets

As with lists, used to store multiple items of data
Does not allow duplicate values

user_input = ""
while user_input != "exit":
    user_input = input("Hey user, enter number of days as a comma separated list and I will convert it to hours!\n")
    list_of_days = user_input.split(", ")
    for num_of_days_element in set(list_of_days):
        validate_and_execute()

Items do not have a defined order
Items cannot be referred to by index
Items cannot be changed, only added/removed
Can access elements in a loop

Built-in Functions

We have global functions
There are also built-in functions on data types

Modules

Logically organize Python code
Contain related code
Definitions: things in a module that can be used in another file
Python has built-in modules

days-to-units.py

from helper import validate_and_execute
# from helper import *
# import helper
# import helper as h

user_input = ""
while user_input != "exit":
    user_input = input("Hey user, enter number of days and conversion unit!\n")
    days_and_unit = user_input.split(":")
    print(days_and_unit)
    days_and_unit_dictionary = {"days": days_and_unit[0], "unit": days_and_unit[1]}
    print(days_and_unit_dictionary)
    print(type(days_and_unit_dictionary))
    validate_and_execute(days_and_unit_dictionary)

Packages: PyPI and pip

PyPI: Python package index
Module vs package
- Module is a Python file that contains functions and vars that you can use
- A package is a collection of Python modules
  - Package must include an init.py file
  - The init file distinguishes a package from a directory
Library: multiple packages together
pip
- Package manager tool for python
- pip is packaged with Python

Project Automation with Python Spreadsheet

Exercises
- List each company with respective product count
- List products with inventory less than 10
- List each company with respective total inventory value
- Write to Spreadsheet: Calculate and write inventory value for each product into spreadsheet

pip install openpyxl

Exercise 1

solution.py

import openpyxl

inv_file = openpyxl.load_workbook("inventory.xlsx")
product_list = inv_file['Sheet1']

products_per_supplier = {}

for product_row in range(2, product_list.max_row + 1):
    supplier_name = product_list.cell(product_row, 4).value
    if supplier_name in products_per_supplier:
        products_per_supplier[supplier_name] = products_per_supplier[supplier_name] + 1
    else:
        products_per_supplier[supplier_name] = 1

print(products_per_supplier)

Access dictionary

products_per_supplier[supplier_name] = products_per_supplier.get(supplier_name) + 1

Exercise 2

import openpyxl

inv_file = openpyxl.load_workbook("inventory.xlsx")
product_list = inv_file['Sheet1']

products_per_supplier = {}
total_value_per_supplier = {}

for product_row in range(2, product_list.max_row + 1):
    supplier_name = product_list.cell(product_row, 4).value
    inventory = product_list.cell(product_row, 2).value
    price = product_list.cell(product_row, 3).value

    # calculation number of products per supplier
    if supplier_name in products_per_supplier:
        current_num_products = products_per_supplier.get(supplier_name)
        products_per_supplier[supplier_name] = current_num_products + 1
    else:
        products_per_supplier[supplier_name] = 1

    # calculation total value of inventory per supplier
    if supplier_name in total_value_per_supplier:
        current_total_value = total_value_per_supplier.get(supplier_name)
        total_value_per_supplier[supplier_name] = current_total_value + inventory * price
    else:
        total_value_per_supplier[supplier_name] = inventory * price


print(products_per_supplier)
print(total_value_per_supplier)

Exercise 3

import openpyxl

inv_file = openpyxl.load_workbook("inventory.xlsx")
product_list = inv_file['Sheet1']

products_per_supplier = {}
total_value_per_supplier = {}
products_under_10_inv = {}

for product_row in range(2, product_list.max_row + 1):
    supplier_name = product_list.cell(product_row, 4).value
    inventory = product_list.cell(product_row, 2).value
    price = product_list.cell(product_row, 3).value
    product_num = product_list.cell(product_row, 1).value

    # calculation number of products per supplier
    if supplier_name in products_per_supplier:
        current_num_products = products_per_supplier.get(supplier_name)
        products_per_supplier[supplier_name] = current_num_products + 1
    else:
        products_per_supplier[supplier_name] = 1

    # calculation total value of inventory per supplier
    if supplier_name in total_value_per_supplier:
        current_total_value = total_value_per_supplier.get(supplier_name)
        total_value_per_supplier[supplier_name] = current_total_value + inventory * price
    else:
        total_value_per_supplier[supplier_name] = inventory * price

    # logic products with inventory less than 10
    if inventory < 10:
        products_under_10_inv[int(product_num)] = int(inventory)


print(products_per_supplier)
print(total_value_per_supplier)
print(products_under_10_inv)

Exercise 4

import openpyxl

inv_file = openpyxl.load_workbook("inventory.xlsx")
product_list = inv_file['Sheet1']

products_per_supplier = {}
total_value_per_supplier = {}
products_under_10_inv = {}

for product_row in range(2, product_list.max_row + 1):
    supplier_name = product_list.cell(product_row, 4).value
    inventory = product_list.cell(product_row, 2).value
    price = product_list.cell(product_row, 3).value
    product_num = product_list.cell(product_row, 1).value
    inventory_price = product_list.cell(product_row, 5)

    # calculation number of products per supplier
    if supplier_name in products_per_supplier:
        current_num_products = products_per_supplier.get(supplier_name)
        products_per_supplier[supplier_name] = current_num_products + 1
    else:
        products_per_supplier[supplier_name] = 1

    # calculation total value of inventory per supplier
    if supplier_name in total_value_per_supplier:
        current_total_value = total_value_per_supplier.get(supplier_name)
        total_value_per_supplier[supplier_name] = current_total_value + inventory * price
    else:
        total_value_per_supplier[supplier_name] = inventory * price

    # logic products with inventory less than 10
    if inventory < 10:
        products_under_10_inv[int(product_num)] = int(inventory)

    # add value for total inventory price
    inventory_price.value = inventory * price

print(products_per_supplier)
print(total_value_per_supplier)
print(products_under_10_inv)

inv_file.save("inventory_with_total_value.xlsx")

Objects and Classes

user.py

class User:
    def __init__(self, user_email, name, password, current_job_title):
        self.email = user_email
        self.name = name
        self.password = password
        self.current_job_title = current_job_title

    def change_password(self, new_password):
        self.password = new_password

    def change_job_title(self, new_job_title):
        self.current_job_title = new_job_title

    def get_user_info(self):
        print(f"User {self.name} currently works as a {self.current_job_title}. You can contact them at {self.email}")

main.py

from user import User
from post import Post

app_user_one = User("nn@nn.com", "Nana Janashia", "pwd1", "DevOps Engineer")
app_user_one.get_user_info()

app_user_two = User("aa@aa.com", "James Bond", "supersecret", "Agent")
app_user_two.get_user_info()

new_post = Post("on a secret mission today", app_user_two.name)
new_post.get_post_info()

API Request to Gitlab

Makes use of HTTP requests and responses
We'll use GitLab's API
We can use the requests library

pip install requests

main.py

import requests
response = requests.get("https://gitlab.com/api/v4/users/nanuchi/projects")
my_projects = response.json()

# print the whole objects list
print(my_projects)
print(type(my_projects))

# print just the names and urls
for project in my_projects:
    print(f"Project Name: {project['name']}\nProject Url: {project['http_url_to_repo']}\n")