Posted on

part 2 – Lift and Shift – Rapid IaaS deployment using Python Script on Azure

In this Post I am going to play with Azure SDK to get a script deploying my infra.

Before, starting, you need to prepare your server :

  • Install Python SDK : 
    pip install azure
  • create credentials for your App. The procedure is well explained on MS site here.

after those 2 step, it is time to write !!

To be honest after using python code to deploy, I find it not very friendly, too much variables to handle and too many lines. Without the debug part, it took me 400 lines of code in fully used SDK  . See my Github for this. Then I tried an hybrid approach made of CLI and Python, which was better ( less line and easier to handle variables and threading ).

Again, you can find it on Github too.

import subprocess
import time
import os
import json
import sys
import logging
import time
import datetime

def create_RG(RGROUP,location):
 # Create a resource group VVV
 cmd_rg_create='az group create'+' '+'--resource-group'+' '+RGROUP+' '+'--location'+' '+ location
 print cmd_rg_create
 #os.system(cmd_rg_create)

def command_create_NSG (RGROUP,tier): 
 name_NSG = "NGS-generic-linux-N-tier-%s" % (tier)
 cmd_nsg='az network nsg create'+' '+'--resource-group'+' '+RGROUP+' '+'--name'+' '+ name_NSG
 print cmd_nsg
 #os.system(cmd_nsg)

def command_create_NSG_rule (RGROUP,tier,port_to_process,direction_rule,priority):
 # Create a network security group rule for port 22.
 name_NSG = "NGS-generic-linux-N-tier-%s" % (tier)
 name_NSG_rule = "%s-rule-%s_%s" % (name_NSG,port_to_process,direction_rule)
 cmd_part_1 ='az network nsg rule create'+' '+'--resource-group'+' '+RGROUP+' '+'--nsg-name'+' '+ name_NSG
 cmd_part_2 ='--name'+' '+name_NSG_rule+' '+'--protocol tcp --direction'+' '+direction_rule
 cmd_part_3 ='--source-address-prefix'+' '+"'*'"+' '+'--source-port-range'+' '+"'*'"+' '+'--destination-address-prefix'+' '+"'*'"
 cmd_part_4 ='--destination-port-range'+' '+str(port_to_process)+' '+'--access allow --priority'+' '+ priority
 cmd_NSG_rule = cmd_part_1+' '+cmd_part_2+' '+cmd_part_3+' '+cmd_part_4
 print cmd_NSG_rule
 #os.system(cmd_NSG_rule)

def create_list_NSG(list_of_port,direction_rule,priority_root):
 INC_NSG_RULES = 0
 for port_to_process in portlist_NSG_FRONT:
 priority="%s%s" % (priority_root,INC_NSG_RULES)
 command_create_NSG_rule (RGROUP,tier,port_to_process,direction_rule,priority)
 INC_NSG_RULES+=1

def create_vnet(RGROUP,location,vnet_general_name,vnet_general_IP_RANGE):
 # Create a virtual network VVV
 cmd_part_1 = 'az network vnet create'+' '+'--resource-group'+' '+RGROUP+' '+'--location'+' '+ location
 cmd_part_2 ='--name'+' '+vnet_general_name+' '+'--address-prefix'+' '+vnet_general_IP_RANGE
 cmd_vnet_create = cmd_part_1+' '+cmd_part_2
 print cmd_vnet_create
 #os.system(cmd_vnet_create)

def create_subnet(subnet_level,tier,RGROUP,vnet_general_name,IP_RANGE):
 name_subnet = "subnet-%s" % (subnet_level)
 name_NSG = "NGS-generic-linux-N-tier-%s" % (tier)
 cmd_part_1 ='az network vnet subnet create'+' '+'--address-prefix'+' '+IP_RANGE+' '+'--name'+' '+ name_subnet
 cmd_part_2 ='--resource-group'+' '+RGROUP+' '+'--vnet-name'+' '+vnet_general_name
 cmd_part_3 ='--network-security-group'+' '+name_NSG
 cmd_create_subnet = cmd_part_1+' '+cmd_part_2+' '+cmd_part_3
 print cmd_create_subnet
 #os.system(cmd_create_subnet)


RGROUP="RGMOURAD_0"
#CREATE Ressources Goup
location= "eastus"
create_RG(RGROUP,location)
#CREATE General VNET
vnet_general_name = "vnet-root_2 "
vnet_general_IP_RANGE = "10.100.0.0/14"
create_vnet(RGROUP,location,vnet_general_name,vnet_general_IP_RANGE)
#CREATE First tier NSG and Subnet 
tier="1"
command_create_NSG(RGROUP,tier)
portlist_NSG_FRONT=[22,80,8080,3306]
direction_rule ="inbound"
priority_root="100"
create_list_NSG(portlist_NSG_FRONT,direction_rule,priority_root)
direction_rule ="Outbound"
priority_root="200"
create_list_NSG(portlist_NSG_FRONT,direction_rule,priority_root)
subnet_level="primary"
IP_RANGE="10.100.1.0/24"
create_subnet(subnet_level,tier,RGROUP,vnet_general_name,IP_RANGE)
Posted on

Coding – Resizing Pics using Python.

Hi,

I was looking for a code to resize my picts, first code was to resize pictures by reducing the height and width using a ratio of X.  Then I thought about changing the code to set the threshold as a size limit… meaning recursively reducing size until I go below the 1MB size per picture ( Why 1 MB ? because of WordPress… 😉 ).

Here is a code that means to reduce the size. It can be improved to parallelize the treatment of pics ( currently only serial treatment ), I will see if I have time for that…

EDIT: I added threading using library in python. So Now, images are processed in a parallel way.

How it works ?

First make sure you have “PIL” installed , else, just do :

sudo apt-get install python-imaging

just call the script and do not forget the “/” at the end of the directory where the sources are stored :

 python image_resizer.py source_images/

just call the script and do not forget the “/” at the end of the directory where the sources are stored :

here is the link to download the code :  https://rgcloudmouradgeneralpurp.blob.core.windows.net/exchangecontainermourad/image_resizer.py

here is an example of result.

 DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAFA.PNG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 1333433 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 1333433 Bytes
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAFA.PNG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 1196675 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 1196675 Bytes
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAFA.PNG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 1052949 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 1052949 Bytes
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAFA.PNG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 910881 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAFA.PNG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 910881 Bytes
DEBUG ******************************** IMAGE IS SMALLER THAN THRESHOLD 1048576 Bytes
DEBUG ******************************** strimage_to_process is SAFA.PNG
DEBUG ******************************** path_to_process_image is source_images/SAFA.PNG
DEBUG ******************************** INC_NEW_NAME is TREATED_
DEBUG ******************************** NEW_NAME is dest_images/TREATED_SAFA.PNG
DEBUG ******************************** image_written
IMAGE SAM_0305.JPG WILL BE REZIED TO FIT THRESHOLD
DEBUG ******************************** Image_Current_Size is 6086874 Bytes
DEBUG ******************************** initial_ratio_image_source_dest_size is 5
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAM_0305.JPG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 1605512 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 1605512 Bytes
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAM_0305.JPG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 1351356 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 1351356 Bytes
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAM_0305.JPG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 1127460 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 1127460 Bytes
DEBUG ******************************** IMAGE IS BIGGER THAN THRESHOLD 1048576 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** buffer_for_image IS <open file 'source_images/SAM_0305.JPG', mode 'r' at 0x7f606ee0e270> Bytes
DEBUG LOOP images_reducer ******************************** TEMP_NAME IS temp_reducing_images/TEMP_SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** Image_local_Current_Size IS 928476 Bytes
DEBUG LOOP images_reducer ******************************** path_to_process_image IS source_images/SAM_0305.JPG Bytes
DEBUG LOOP images_reducer ******************************** REPLACED ORGINAL FILE 928476 Bytes
DEBUG ******************************** IMAGE IS SMALLER THAN THRESHOLD 1048576 Bytes
DEBUG ******************************** strimage_to_process is SAM_0305.JPG
DEBUG ******************************** path_to_process_image is source_images/SAM_0305.JPG
DEBUG ******************************** INC_NEW_NAME is TREATED_
DEBUG ******************************** NEW_NAME is dest_images/TREATED_SAM_0305.JPG
DEBUG ******************************** image_written
DEBUG ******************************** ALL IMAGES HAVE BEEN RESIZED... MOVING DATA BACK TO INITIAL FOLDER
DEBUG ******************************** ALL IMAGES HAVE BEEN MOVED BACK.... CLEANING...

…