Blog Archives

TechNet on Tour – Increasing efficiency and scalability with Windows Server 2016 and Microsoft cloud

Want to learn how to evolve your datacenter into a true hybrid cloud model to achieve greater efficiency, flexibility and scalability? If you are around Redmond, WA – sign up for free one-day training event – https://www.microsoftevents.com/profile/form/index.cfm?PKformID=0x111759537c

This training will cover:

  • Confidently virtualize your mission critical workloads on premises with Hyper-V, and in the cloud with Microsoft Azure
  • Implement new software-defined storage and network architectures to increase storage performance, flexibility and redundancy
  • Get the smallest footprint, reduced patching, and most cloud-optimized OS performance available, with Nano Server
  • Extend datacenter resources by using virtual networking to bridge the gap between your on-premises infrastructure and Microsoft Azure
  • Explore single-click deployment of complex, multi-tier apps across VMs, storage, and networking with the new Azure Resource Manager

I will be delivering one of the sessions. See you there at the venue. J

Docker Swarm Cluster using Azure Container Services

Azure Container Service (ACS) is an Azure Compute resource provider to create and manage a cluster of virtual machines that act as container hosts together with a number of “master” machines that are used to orchestrate your application containers. Currently ACS is in preview, and my subscriptions got access to the preview service last week. I will be exploring ACS and blogging about them.

ACS works with Apache Mesos as well as Docker Swarm. In this article, I am sharing my experience on creating Docker Swarm cluster on ACS. For build the service, I used the guidance provided at https://github.com/rgardler/azure-quickstart-templates/blob/acs/acs-docs/101%20Creating%20a%20Service%20using%20the%20Portal.md as the primary reference.

Prerequisites

  • Microsoft Azure Subscription enabled with ACS preview: If you need access, please request here – http://aka.ms/acspreview.
  • Client (Git or Open SSH) to generate RSA Keys

Setup Sequence

  • Create SSH RSA Public Key
  • Deploy Swarm ACS Template
  • Explore Resources in Container Service

Create SSH RSA Public Key

I used Git Client on my Windows Laptop to create the SSH key pair.

# Use ssh-keygen to create RSA key pair

# id_rsa and id_rsa.pub files are created in ~/.ssh folder

ssh-keygen –t rsa –b 2048

Open id_rsa.pub file, which has three parts (ssh-rsa <key> user@machinename). We need to copy all these content and provide as the SSHRSAPUBLICKEY, while deploying the template for ACS.

Deploy Swarm ACS Template

Open Browser and navigate to https://github.com/rgardler/azure-quickstart-templates/tree/acs/acs-swarm-full-template.

Click on Deploy to Azure button.

Provide parameter values for the custom deployment and click OK.

  • DNSNAMEPREFIX: Unique DNS prefix
  • AGENTCOUNT: You can choose 1 through 40. Provide 3.
  • MASTERCOUNT: 1, 3 or 5 are allowed now. Choose 3.
  • AGENTVMSIZE: For experiments, choose lower size (helps with reduced cost)
  • SSHRSAPUBLICKEY: Ensure that you copy all the three parts created in previous step and provide here.

Choose to Create a new Resource Group, name it ACSExperiment.

Ensure that you choose Japan East
as the resource group location as this is the only supported region for ACS now.

Click on Legal Terms, review the terms and click Create.

Click on Create from the custom deployment blade.

Wait for the deployment to complete.

Explore Resources in Container Service

Once the template deployment is complete, open the Resource Group, ACSExperiment. You can see that there are four categories of resources.

  • Azure Container Service Resource – Which manages all other resources
  • A number of agent nodes with in an availability set for agents
  • A number of master nodes with in an availability set for masters
  • Availability set (one for agents and another for master)

In the next article, I will be going through in-detail on exploring the above resources, followed by publishing apps to Swarm cluster. J

Microsoft Azure Training & Workshops to Solution Architects from various Global System Integrators

Last two weeks, I have travelled to Bangalore, India to provide trainings to Architects from various global system integrators on Microsoft Azure technology. During this travel, I have delivered sessions / workshops on the following:

  • Architecting Big Data & Analytics Solutions: The proliferation of customer data and the desire for insights not previously attainable has created a new industry focused on analyzing massive amounts of data. The cost of processing huge amounts of data does not need to be prohibitive when you can take advantage of a scalable cloud platform. We will discuss the Lambda architecture to take advantage of both batch and stream processing of data and will show various Azure services such as Azure Data Lake, Azure Data Factory, Azure Stream Analytics, Azure Machine Learning, and Power BI to provide a solution to unlock insights into data.

 

  • Architecting Modern Cloud Applications: Modern cloud applications offer end-user experiences and features that transcend traditional on-premises applications. For example, there are often multiple database technologies supporting today’s modern app. Embrace the notion of polyglot persistence and see how RDBMS’s, NoSQL databases, and caches can be combined to deliver robust end-user experiences. Modern cloud apps today are also expected to be reachable regardless of the platform or device. See how Azure’s App Service can be used to expand the reach of your solutions. Modern cloud apps are also not always just platform-as-a-service solutions. Learn about modern architecture patterns enabled by Azure that span infrastructure-as-a-service and platform-as-a-service deployments.

 

  • Architecting Global Scale Web and Mobile Solutions: One of the main benefits of the Microsoft Azure platform is its massive scale. Microsoft Azure runs on a massive network of over 19 regions worldwide. This global reach makes it possible to create solutions on a global scale that meet your demanding performance requirements. We will discuss architectural patterns that lend themselves to global geo-located solutions and the specific features of Microsoft Azure that enable geo-replicated data.

 

  • Dev Ops: Enterprise IT organizations are increasingly driven by business demands for faster, better solutions delivered more quickly than ever before. DevOps has emerged as a trend to help organizations evolve to better collaborate between development and operations teams for addressing these solution delivery challenges. While a DevOps transformation can take time to fully realize within an organization’s people and processes, there’s opportunities to accelerate the path towards DevOps by smartly leveraging cloud capabilities that can reduce infrastructure complexity, simplify release management, and support end-to-end application visibility through all stages of a solution’s lifecycle. This session will discuss the features in Microsoft Azure that help to realize this transformation and how the platform enables integration with existing assets.

 

  • Case Study – Internet of Things (IoT)
  • Case Study – Refactoring Multi-Tier Solutions
  • Case Study – Lift and Shift
  • Case Study – Identity and Access Management
  • Case Study – Networking

Setup Jenkins Job to start on Git Commit

 

This is the fifth post in the series of ‘Continuous Integration and Delivery with Docker Trusted Registry on Microsoft Azure’. This is a step by step guidance for implementing http://www.docker.com/sites/default/files/UseCase/RA_CI%20with%20Docker_08.25.2015.pdf on Microsoft Azure.

You can read the previous post in this series at https://manesh.me/2015/11/15/continuous-integration-and-delivery-with-docker-trusted-registry-on-microsoft-azure/. In the previous post, we have created a job in Jenkins to have on-demand start based on Git source repository. In this post, we will explore, how we can setup Git/Jenkins to work together to invoke the job for every check-in. To configure and try out this post, you will need write permission to Git repository. I am using a forked repository https://github.com/Manesh-R/hello-jenkins, for this post.

Prerequisites

  • Microsoft Azure Subscription
  • Followed steps in all prior posts
  • Write permission on Git repository (Forking the Git project will be the easiest way)

Setup Sequence

  • Configure GitHub Service Hook
  • Configure Build Trigger on Git Commit
  • Test build trigger with a commit to Git repository

Configure GitHub Service Hook

Open browser and navigate to your GitHub repository (https://github.com/Manesh-R/hello-jenkins). Note that you need admin permissions on the repository.

Click on Settings, then Webhooks & services and then Add service

Search for Jenkins and click on Jenkins (GitHub plugin)

Update Jenkins hook URL, keep service Active and click Add service.

Now for every commit to the repository, service hook will send notification to Jenkins.

Configure Build Trigger on Git commit

Open browser and navigate to http://mme-j-master.southcentralus.cloudapp.azure.com:8080/

Click on job dtr-cicd.

Click on Configure

In project details select Build when a change is pushed to GitHub in Build Triggers and click Save.

Test build trigger with a commit to Git repository

Open browser and navigate to https://github.com/Manesh-R/hello-jenkins/edit/master/README.md

Make a minor modification to the file content and commit changes directly to master.

Now you should see a build automatically triggering on Jenkins server based on the service hook notification. J

Summary

With this we have gone through the complete cycle of setting up a continuous integration / delivery pipeline using Docker & Jenkins on Microsoft Azure. If you have any questions, add a comment to the posts. Also, if you are interested in getting notified for new blog posts, follow me on Twitter – http://www.twitter.com/ManeshRavendran.

Setup Jenkins Job with Git Source

 

This is the fourth post in the series of ‘Continuous Integration and Delivery with Docker Trusted Registry on Microsoft Azure’. This is a step by step guidance for implementing http://www.docker.com/sites/default/files/UseCase/RA_CI%20with%20Docker_08.25.2015.pdf on Microsoft Azure.

You can read the previous post in this series at https://manesh.me/2015/11/20/setup-jenkins-slave-on-microsoft-azure-for-cicd-with-dtr/. You can find all posts in this series at https://manesh.me/2015/11/15/continuous-integration-and-delivery-with-docker-trusted-registry-on-microsoft-azure/.

Prerequisites

  • Microsoft Azure Subscription

Setup Sequence

  • Install and Configure GitHub plugin
  • Setup Build Job

Install and Configure GitHub plugin

Open browser and navigate to http://mme-j-master.southcentralus.cloudapp.azure.com:8080/

Click on Manage Jenkins

Click on Manage Plugins

Click Available, type GitHub in the filter, choose GitHub Plugin from the results, and click Download now and install after restart

Wait for install and upgrade of plugins to be complete.

Setup Build Job

From Jenkins dashboard, click New Item, provide item name ‘dtr-cicd‘, choose Freestyle project and click OK

For the project details, provide following information and click Save

  • GitHub Project: https://github.com/mikegcoleman/hello-jenkins/
  • Restrict where this project can be run (Choose this option and provide docker as label expression)
    • docker
  • Source Code Management: Git
  • Repository URL: https://github.com/mikegcoleman/hello-jenkins.git
  • Build > Add Build Step > Execute Shell

    #build docker image

    sudo docker build –pull=true -t mme-dtr.southcentralus.cloudapp.azure.com/hello-jenkins:$GIT_COMMIT .

     

    #test docker image

    sudo docker run -i –rm mme-dtr.southcentralus.cloudapp.azure.com/hello-jenkins:$GIT_COMMIT ./script/test

     

    #push docker image

    sudo docker push mme-dtr.southcentralus.cloudapp.azure.com/hello-jenkins:$GIT_COMMIT

Click on Build Now from the job page.

You should see that project builds successfully. J

Note: You can configure build schedule to be initiated when a change is pushed to GitHub.

Setup Jenkins Slave on Microsoft Azure for CICD with DTR

This is the third post in the series of ‘Continuous Integration and Delivery with Docker Trusted Registry on Microsoft Azure’. This is a step by step guidance for implementing http://www.docker.com/sites/default/files/UseCase/RA_CI%20with%20Docker_08.25.2015.pdf on Microsoft Azure.

You can read the previous post in this series at https://manesh.me/2015/11/15/continuous-integration-and-delivery-with-docker-trusted-registry-on-microsoft-azure/.

Prerequisites

  • Microsoft Azure Subscription

Setup Sequence

  • Create virtual machine for Jenkins Slave
  • Assign DNS name for Jenkins Slave machine
  • Install JRE on Jenkins Slave
  • Install and Configure Docker Engine on Jenkins Slave
  • Register slave node in Jenkins Master

Create virtual machine for Jenkins Slave

Open browser and navigate to https://portal.azure.com/

From Azure Portal, click New, then Compute, then Marketplace

Type Ubuntu Server in the search box and press Enter, then click on ‘Ubuntu Server 14‘ from the search results

Read virtual machine image description, choose Resource Manager as the deployment model and click Create

Note: It is important to choose ‘Resource Manager’ deployment model, to have this created in the same resource group of DTR and Jenkins Master.

Provide basic configuration details for the virtual machine (you can change details as per your environment) and click OK

  • Name: mme-j-slave01
  • User name: cicdagent
  • Authentication Type: Password
  • Password: Azure101!
  • Resource Group: DTR
    • Select existing Resource Group, you created for DTR virtual machine
  • Location: South Central US

For size configuration, choose D1 Standard and click Select

Choose default values for optional configurations, and click OK

Review summary details and click OK

Wait for virtual machine to get into state ‘Running’.

Assign DNS Name for Jenkins Slave

Follow the similar steps in previous post to create a DNS name label for the virtual machine. In my environment FQDN of the virtual machine is mme-j-slave01.southcentralus.cloudapp.azure.com

Install JRE on Jenkins Slave

Use SSH to connect to Jenkins Slave machine and execute the following commands.

ssh cicdagent@mme-j-slave01.southcentralus.cloudapp.azure.com

# install jre package on the virutal machine

sudo apt-get install default-jre

Install and Configure Docker Engine on Jenkins Slave

Use SSH to connect to Jenkins Slave machine and execute the following commands.

ssh cicdagent@mme-j-slave01.southcentralus.cloudapp.azure.com

# download and install CS Engine

curl -s https://packagecloud.io/install/repositories/Docker/cs-public/script.deb.sh | sudo bash

sudo apt-get install docker-engine-cs

Since we do not have a valid SSL certificate on Docker Trusted Registry server for HTTPS, we need to ensure that docker service starts with ‘–insecure-registry mme-dtr.southcentralus.cloudapp.azure.com’ option. For this we need to edit /etc/default/docker file.

You can use vim editor to make required changes on the file. To start inserting text in vim, press the ‘i’ key to enter insert mode. When you have finished editing, press ESC to exit insert mode, then type ‘:wq’ to save and exit the editor.

sudo vim /etc/default/docker

Once the file is updated, reboot the Jenkins Slave virtual machine.

sudo reboot

Register slave node in Jenkins Master

Open browser and navigate to http://mme-j-master.southcentralus.cloudapp.azure.com:8080

Click on Manage Jenkins

Click on Manage Nodes

Click on New Node

Enter mme-j-slave01 as the node name, choose option dump slave and click OK

On node details page, provide slave node information. Click on Add button for credentials and create a username with password credential. It is important to keep label for node as docker. This label is used later to associate Jenkins Job to the slave node.

You should see that the node is added and is in active state.

Next Blog

You can read the next post in this blog series here, <TODO>.

Setup Jenkins Master on Microsoft Azure for CICD with DTR

This is the second post in the series of ‘Continuous Integration and Delivery with Docker Trusted Registry on Microsoft Azure’. This is a step by step guidance for implementing http://www.docker.com/sites/default/files/UseCase/RA_CI%20with%20Docker_08.25.2015.pdf on Microsoft Azure.

You can read all the posts in this series at https://manesh.me/2015/11/15/continuous-integration-and-delivery-with-docker-trusted-registry-on-microsoft-azure/.

Prerequisites

  • Microsoft Azure Subscription

Setup Sequence

  • Create virtual machine for Jenkins Master
  • Assign DNS name for Jenkins Master virtual machine

Create virtual machine for Jenkins Master

Open browser and navigate to https://portal.azure.com/

From Azure Portal, click New, then Compute, then Marketplace

Type Docker Jenkins in the search box and press Enter, then click on ‘Jenkins‘ from publisher ‘Docker‘ in the search results

Read virtual machine image description and click on Create

Provide basic configuration details for the virtual machine (you can change details as per your environment) and click OK

  • Name: mme-j-master
  • User name: cicdagent
  • Authentication Type: Password
  • Password: Azure101!
  • Resource Group: DTR
    • Select existing Resource Group, you created for DTR virtual machine
  • Location: South Central US

For size configuration, choose D1 Standard and click Select

Choose the default Settings for ‘Configure optional features‘ and click OK

Review the summary details and click OK

Initial virtual machine will show status as ‘Deploying‘, wait until it turns to ‘Running‘. I have seen it taking more than couple of minutes to deploy and configuring everything before getting into running state.

Assign DNS name for Jenkins Master virtual machine

Follow the similar steps in previous post to create a DNS name label for the virtual machine. In my environment FQDN of the virtual machine is mme-j-master.southcentralus.cloudapp.azure.com

Once virtual machine is in running state and DNS name label is assigned, browse to
http://mme-j-master.southcentralus.cloudapp.azure.com:8080

Next Blog

You can read the next post in this blog series here, https://manesh.me/2015/11/20/setup-jenkins-slave-on-microsoft-azure-for-cicd-with-dtr/.

Setup Docker Trusted Registry on Microsoft Azure

This is the first post in the series of ‘Continuous Integration and Delivery with Docker Trusted Registry on Microsoft Azure’. This is a step by step guidance for implementing http://www.docker.com/sites/default/files/UseCase/RA_CI%20with%20Docker_08.25.2015.pdf on Microsoft Azure.

You can read all the posts in this series at https://manesh.me/2015/11/15/continuous-integration-and-delivery-with-docker-trusted-registry-on-microsoft-azure/.

Prerequisites

  • Microsoft Azure Subscription

Setup Sequence

  • Create virtual machine for Docker Trusted Registry
  • Assign DNS name for DTR virtual machine
  • Download DTR License from Docker Hub
  • Update settings of DTR virtual machine

Create virtual machine for Docker Trusted Registry

Open browser and navigate to https://portal.azure.com/

From Azure Portal, click New, then Compute, then Marketplace

Type Docker in the search box and press Enter, then click on ‘Docker Trusted Registry‘ from the search results

Read the description of marketplace virtual machine image for DTR and click Create

Provide basic configuration details for the virtual machine (you can change details as per your environment) and click OK

  • Name: mme-dtr
  • User name: cicdagent
  • Authentication Type: Password
  • Password: Azure101!
  • Resource Group: DTR
  • Location: South Central US

For size configuration details, choose D2 Standard and click Select

Choose default settings for optional features, and click OK

Review summary details and click OK

Review buy / offer details and click Purchase

Now you will see that Docker Trusted Registry is Deploying. Wait for the virtual machine to be in Running state.

Assign DNS name for DTR virtual machine

Click on Virtual Machines (remember, we created the virtual machine with Resource Manager as the deployment model) and click on mme-dtr

Click on Public IP Address

Click on Configuration, enter mme-dtr for DNS name label and click Save. Note that sometimes dns names might be already reserved by someone else, so choose different name if the default vm name is not available.

Ensure you can browse to the virtual machine with public DNS name. In this case it is https://mme-dtr.southcentralus.cloudapp.azure.com. Ignore the certificate error.

Download DTR License from Docker Hub

Open browser and navigate to https://hub-beta.docker.com/enterprise/

Login to the portal. If you do not have an account, sign up first and login to the portal.

Once logged in, navigate to https://hub.docker.com/enterprise/ and click on Get Stated for Trial. Note that based on your requirement, you can choose other subscriptions as well. Also, however we started on hub-beta.docker.com, it redirects back to hub.docker.com

Private your contact information and click on ‘Start Your Free Trial’

Click on down arrow near your username, and click on Settings

Click on Licenses and then on Download icon

Save the downloaded docker_subscription.lic file in your local machine.

Update settings of DTR virtual machine

Open browser and navigate to https://mme-dtr.southcentralus.cloudapp.azure.com/admin/

Click on the warning shown on the dashboard page.

Provide fully qualified domain name and click Save and restart.

Note: I was not able to get the save button working properly on Internet Explorer. It worked well for me on Chrome. Also, server might take about 30 seconds to reboot.

Wait for 30 seconds, refresh the browser and ensure that the ‘domain name missing‘ warning is no longer there.

Click on the ‘Unlicensed copy’ warning message

Click on Choose File, select the downloaded license file from local machine and click on Save and restart

Once license is applied successfully, you should see the Tier getting updated to Trial

Next Blog

You can read the next post in this blog series here, https://manesh.me/2015/11/18/setup-jenkins-master-on-microsoft-azure-for-cicd-with-dtr/.

Continuous Integration and Delivery with Docker Trusted Registry on Microsoft Azure

Docker enables developers and sysadmins to build, ship and run distributed applications anywhere. Use cases are a great way to adopt technology to tackle specific technical or business challenges your team is trying to solve. Docker has provided guidance and a reference architecture for CI/CD with Docker. You can find it at http://www.docker.com/sites/default/files/UseCase/RA_CI%20with%20Docker_08.25.2015.pdf

In this series of blog posts, I will be exploring and implementing this RA fully on Microsoft Azure.

You can download all the original word documents I used to author this blog series from https://gallery.technet.microsoft.com/Continuous-Integration-and-c5ecb6a8. If you have questions, you can add comments to this blog. If you liked my blog series, you can follow me on Twitter at https://www.twitter.com/ManeshRavendran/

Getting started with Chef on Microsoft Azure

You can find an initial set of blog posts to help you get started with Chef on Microsoft Azure here. If you are new to Chef, I recommend you to go through some of the Chef tutorials https://learn.chef.io/. Objective of these posts are not to go deep into Chef, but to help you setup your whole set of machines (Chef Workstation, Chef Server, Chef Node) on Microsoft Azure.

https://manesh.me/2015/10/17/setup-chef-workstation-on-microsoft-azure/

  • Create a virtual machine for workstation
  • Install Chef Development Kit
  • Learn Chef Basics on Ubuntu

https://manesh.me/2015/10/19/setup-chef-server-on-microsoft-azure/

  • Create Virtual Machine
  • Set DNS Name Label for virtual machine
  • Configure Chef Server
  • Create Organization
  • Establish Connectivity from Workstation

https://manesh.me/2015/10/20/setup-chef-node-on-microsoft-azure/

  • Install Azure CLI & Login
  • Create Virtual Machine for Node
  • Bootstrap & Verify Node

https://manesh.me/2015/10/23/writing-cookbook-on-microsoft-azure/

  • Create cookbook on workstation
  • Upload cookbook to server
  • Add cookbook to run list of node
  • Run chef-client on node
  • Enable endpoint using Azure CLI

In the next set of blogs, I will walk you through in detail on various Azure Knife commands as well as using Chef along with Azure Resource Manger templates.

If you have feedback or questions, write a comment for the blog.

hyper-v.nu

My views on technology

yungchou

witnessing a clear cloudy day

Virtualization and some coffee

My views on technology

Thomas Maurer

Cloud and Virtualization

Cloud Administrator

Begin Your Journey to the Cloud with the Cloud Administrator