Azure

Azure Cosmos DB–Reserved Capacity

October 7, 2018 Azure, Azure Cosmos DB, Billing, CosmosDB, Enterprise Agreement, Infrastructure, Microsoft, Pay-as-you-go No comments

Azure Cosmos DB is a planet scale global document database which have been available for Azure Customers based on pay-as-you-go. Reserved Capacity is a new long term pre-paid billing commitment customers can get a discounted pricing. Azure Cosmos DB reserved capacity helps you save money by pre-paying for Azure Cosmos DB resources for a period of one year or three years.

  • Reserve capacity allows you to get a optimum discount for the throughput(RUs) provisioned for Cosmos DB resources.
  • For Ex: Databases and Containers (tables/collections/graphs).
  • It can significantly reduce your Cosmos DB costs and it enables you to save up to 65 percent on regular prices with one-year or three-year upfront commitment.
  • Reserved capacity provides a billing discount and does not affect the runtime state of your Cosmos DB resources.
  • It is available to all supported APIs (including MongoDB, Cassandra, SQL, Gremlin and Azure Tables) and all regions worldwide.
  • Scope of Reservation:

    A reservation is scoped to a single subscription or shared across all your enrolment:

    • Scoped to a Single Subscription: that means a set of cosmos db account resources such as database or containers within the selected subscription.
    • Shared/Scoped to an enrolment: Billing benefit can be shared across any subscription in the enterprise agreement or pay-as-you-go subscriptions.

    cosmos-db-reserved-capacity

    Who can opt-in for Reserved Capacity?

    This feature is currently only applicable for enrolment (Enterprise Agreement customers) or account (Pay-as-you-go customers). Users with MSDN Subscription benefits are not applicable for the Reserved Capacity benefits.

    You can buy Azure Cosmos DB reserved capacity from the Azure portal.

    1. Sign in to the Azure portal.
    2. Select All services > Reservations > Add and Select Product Type > Cosmos DB

    image

    8fafbeff-f11c-4894-98e0-e03d79d1787d

    For more details review reserved capacity documentation.

    Azure Cosmos DB – 429 Too Many Requests

    October 6, 2018 .NET, Azure, CosmosDB, Document DB, Microsoft, Performance, Reliability, Resilliancy, Scalability, Visual Studio 2017, VisualStudio, VS2017 No comments

    Recently while I was doing Performance Testing in one of the APIs interacting with Cosmos DB, I encountered a problem as Azure Cosmos DB API’s started returning Http Code 429.  Http Status Code 429 indicates that too many request been received or request rate is very large. This error would happen when we have concurrent users trying to write or read from same cosmos db collection.

    Following diagram covers the architecture of the performance test I am performing:

    image

    Based on analysis it found out to be the Throttling happening from Azure Cosmos DB, as we make requests that may use more than provisioned Request Units(RU) per second. We were using default Cosmos DB configuration for a fixed collection of 1000 RU’s per second which is sufficient enough for a 500 reads and 100 writes for a 1 kb file. You can refer more about Request Units from Azure Docs.

    image

     

     

     

    Solution(s):

    1. Now first logical step we can do is to get rid off this error by increasing the Throughput for the collection.  I am going to increase to 10000 RU/s maximum allocatable for a Storage Capacity: Fixed.   This should ideally improve the Throughput for 250 or more virtual users hitting.

    image

    2. Second logical step is to improve the code: Improve the connection parameters in the Document DB SDK –> DocumentDbClient. For this I referred to the Microsoft Docs: Performance tips for Azure Cosmos DB and .NET

    Providing optimum values to the following Properties in RetryOption class   to be passed as parameter to Connection Policy.

    image

     

    In my case I provided a value of 30 to give ultimate results:

    new RetryOptions() { MaxRetryAttemptsOnThrottledRequests = 30, MaxRetryWaitTimeInSeconds = 30  }

    That should resolve most of the 429 issues when dealing with Cosmos DB SDK

    Azure Database for MariaDB: Public Preview

    October 4, 2018 Azure Database for MariaDB, Managed Services, MariaDB, OpenSource No comments

    During Ignite 2018, Microsoft has announced the availability of Maria DB support in Azure Database services. Today it has been opened for Public Preview for all Azure customers.

    mariadbhero

    What is MariaDB?

    MariaDB is a community-developed fork of the MySQL relational database management system intended to remain free under the GNU GPL.Development is led by some of the original developers of MySQL, who forked it due to concerns over its acquisition by Oracle Corporation.Wikipedia

    Azure Database for MariaDB: Public Preview Availability

    The Azure Database for MariaDB service is now available in preview. It offers an enterprise-ready, fully managed database service that based on the Community Edition of MariaDB.

    The service features open-source compatibility, built-in high availability, dynamic scaling, and flexible pricing. Customers can lift and shift to the cloud and use languages and frameworks of their choice, leveraging the power of MariaDB running on Azure.

    To learn more about the service, view the service page, pricing, and documentation.

    You can create a MariaDB server by using the Azure portal or Azure CLI.

    More References:

    Azure Cosmos DB–Setting Up New Database using Azure CLI–Sample

    October 1, 2018 Azure, Azure Cosmos DB, Codes, CosmosDB, KnowledgeBase, Microsoft, PowerShell, Windows Azure Development No comments

    Purpose of this article is to help you with few steps of commands to provision a new Azure Cosmos DB database instance through Azure CLI or Azure Cloud Shell.

    Here is the snippet:

     <# 
       This Bash script should help you create a Azure Cosmos DB instance using Azure CLI with bare minimal configuration 
    #>
    
    export ACCOUNT_NAME="thingx-retail-store-db"
    export DB_RESOURCE_GROUP="thingx-dev"
    export DB_LOCATION="southcentralus"
    export DB_NAME="Products" 
    export DB_THROUGHPUT=1000  ## bare minimal for 500 read and 100 write configuration for a 1KB document.
    export DB_COLLECTION_NAME="Groceries"
    
    ##Optional: If resource group does not exist create a new one 
    az group create --name $DB_RESOURCE_GROUP --location $DB_LOCATION
    
    ##1.0 Create the Azure Cosmos DB Account 
    az cosmosdb create --name $ACCOUNT_NAME --kind GlobalDocumentDB --resource-group $DB_RESOURCE_GROUP
    
    ##2.0 Create Products database in the account  
    az cosmosdb database create --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP
    
    ##3.0 Create Groceries collection in Products database
    az cosmosdb collection create --collection-name $DB_COLLECTION_NAME --partition-key-path "/productId" --throughput $DB_THROUGHPUT --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP
    
    
    

     

    For the ease of this article I used Azure Cloud Shell, that you can launch from your azure portal by clicking on the Shell icon on the top portal menu.

    image

    image

    Now you are ready to execute all the commands listed above in the sample bash script

    image

    Create New Azure Cosmos DB Account

     <span data-mce-type="bookmark" style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" class="mce_SELRES_start"></span>
    az cosmosdb create --name $ACCOUNT_NAME --kind GlobalDocumentDB --resource-group $DB_RESOURCE_GROUP

    image

    Create Products Database in the account

    az cosmosdb database create --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP

    image

    Create Groceries collection in Products database

    az cosmosdb collection create --collection-name $DB_COLLECTION_NAME --partition-key-path "/productId" --throughput $DB_THROUGHPUT --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP

    image

    Now if you browse Azure Portal you can see resources created in “thingx-dev” resource group.

    image

    Upon browsing with Data Explorer you can see the Groceries collection inside Products DB.

    image

    So that’s the few silly easy steps to create Cosmos DB database from Azure CLI or Azure Cloud Shell. Hope that makes it easy for you.

    Getting Started with Azure CLI 2.0

    September 30, 2018 Azure, Azure CLI, Azure Cloud Shell, AzureRM.PowerShell, PowerShell No comments

    Older days we used to manage azure resources through AzureRM PowerShell modules . This was very much flexible for any Azure Administrator or Developers to run Automated Deployments to Azure Resource Manager resources.

    Azure CLI  is the next improved version with simplified cmdlets to make life easier and it is cross-platform.

    You can use Azure CLI in two ways:

    1. Azure Portal – Through Azure Cloud Shell
    2. PowerShell module

    Installation Steps:

    • Download Azure CLI designed for Linux/Windows/MacOS based on your OS.
    • Install image and follow the steps.

     

    image

    image

    • Verify the Installation by executing cmdlet [  az –version  ]
    az –-version

    image

    Running the Azure CLI from PowerShell has some advantages over running the Azure CLI from the Windows command prompt, provides additional tab completion features.

    Now let us try logging in to Azure using Azure CLI. There are various ways of logging in, for this article I would try simple web login using az login command.

    Execute the following cmdlet to login to Azure:

    az login

    The Azure CLI will  launch your default browser to open the Azure sign-in page. After a successful sign in, you’ll be connected to your Azure subscription.  If it fails, follow the command-line instructions and enter an authorization code at https://aka.ms/devicelogin.

    Create a azure resource group and verify:

    az group create –name "thingx-dev" –location "southcentralus" 
    az group list --output table

     

    Hope that is helpful for you to get started with Azure CLI. To learn more about Azure CLI cmdlets : https://github.com/Azure/azure-cli

    NDepend–VSTS/Azure DevOps Integration–Part 01

    September 30, 2018 .NET, .NET Core, .NET Framework, Azure DevOps, Best Practices, Code Analysis, Code Quality, Dynamic Analysis, Emerging Technologies, Microsoft, Static Analysis, Tools No comments

    In my previous article I wrote an introductory about NDepend and how it will be useful for Agile Team to ensure code quality.

    In that article we found how we can use NDepend in a developer machine. Now with this article we will familiarize ourselves in using NDepend in your build automation pipeline in your VSTS/Azure DevOps Build Agent.

    There are two types of integration possible for NDepend:

    1. Directly using NDepend Package Extension from VSTS Marketplace
    2. Manual Integration using NDepend Command Line Tool. (This would provide you more control over licensing by setting up the license in your own OnPrem VSTS Build Agent.

    For the interest of this article I will cover the use of VSTS Package Extension and using NDepend Build Task in VSTS Build Pipeline.

    Installation of NDepend Extension for VSTS/Azure DevOps :

    1.) Got to Azure DevOps Market Place:  https://marketplace.visualstudio.com/items?itemName=ndepend.ndependextension

    image

    2.) Click on Get to Install this extension in to your AzureDevOps account and follow the steps. For the demo purpose I am starting with 30 day free trial, otherwise you can go ahead and buy the full license.

    image

    image

    image

    3.) Now when you get back to Azure DevOps project, you can see the NDepend side menu enabled, this is where you would see the report summary of your project.

    image

    Integration NDepend into Azure DevOps Pipeline :

    1.) Select “NDepend Task” and add in to Pipeline

    image

    image

    Note:

    • You can choose to stop the build when at least one quality gate fails.
    • You also need to specify the NDepend project file customized for your project, otherwise NDepend will use their default project file configuration.  Having your own NDepend project file will provide you more control over the policies for the scan.

    Queue a new Build and wait for Build to complete. Now you can see the BuildArtifacts includes all NDepend report file.

    image

    Now you go back to NDepend menu from Left side menu item in Summary Tab. This will provide you detailed view of Technical Debt in your project.

    image

    image

    image

    image

    image

    In the next article I will cover the manual integration steps.