CosmosDB

Azure Cosmos DB – Change feed support(PREVIEW)–available

July 26, 2018 .NET, .NET Core, .NET Framework, Azure, Azure Cosmos DB, C#.NET, CosmosDB, Microsoft No comments

Today Microsoft announced the preview of Change feed Support for Azure Cosmos DB, which allows you to build scalable solutions. By default change feed will be enabled in all the accounts.

Change feed provides an output of sorted list of documents that has been changed in the order in which they are modified by client operations. These changes are persisted, can be processed asynchronously and incrementally, they enables developers to write alternative logic to operate upon these change for generation reports, or invoke another operation such as sending email or audit logs etc.

ChangeFeedProcessor

Start using :

Source:

Azure Cosmos DB – Consistency Levels

June 2, 2018 Azure, CosmosDB, Data Consistancy, Data Integrity, Higher Availability, Microsoft, Reliability, Resilliancy, Scalability No comments

CosmosDB is a planet scale multi model, multi-region NoSQL database service provided as part of Azure Platform. Azure Cosmos DB is designed to provide global distribution for every data model you choose while creating Cosmos DB.  It is promised to provide low latency and various well-defined consistency models to ensure data redundancy and high availability.

In this short diagram I will be covering the different consistency models available with Cosmos DB and their benefits:

image

There said depending on your data criticality and needs of faster accessibility, you can choose between any of the above consistency models. I strongly trusts in session consistency, as it ensures a balance b/w both.  But again it is totally depending on your business case and how critical is your system depends on the accuracy of this data.

Hope you enjoyed this short article!.

Further reads: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels

Azure Cosmos DB – Programatically Connect to a preferred location using the SQL API

May 29, 2018 .NET, Azure, CosmosDB, Microsoft, VisualStudio, Windows, Windows Azure Development No comments ,

Cosmos Db is a multi-region scallable, globally-distributed database solution as part of Microsoft Azure Platform.  With a button click, Azure Cosmos DB enables you to elastically and independently scale throughput and storage across any number of Azure’s geographic regions. It offers throughput, latency, availability, and consistency guarantees with comprehensive service level agreements (SLAs),  that no other database service can offer. [REF]

What is multi-region scalability or global distribution ?

What it means is that once you select this option, and underlying platform will ensure that your main database is replicated across other global regions you have defined.

So when a customer/application requests the data from a certain geo location:

  1. Cosmos Db will serve the data from nearest available regional copy to provide low latency in accessing the database.  Inorder to achieve it is recommended to deploy both the application and Azure Cosmos DB in the regions that correspond.
  2. Incase that nearest available region is not defined, it would serve from nearest available or main copy. This could be East US or West US depending on your deployment decisions.
  3. As BCDR(Business Continuity and Disaster Recovery) plan, Incase main copy is not available, it would faillover to serve the requests from any backup region.  

Benefits?

  • Ensured AVAILABILITY @ 99.99% – Azure Cosmos DB offers low latency reads and writes at the 99th percentile worldwide.
  • Faster READS: It ensures that all reads are served from the closest (local) region.  To serve a read request, the quorum local to the region in which the read is issued is used.
  • Reliable WRITES: The same applies to writes. A write is acknowledged only after a majority of replicas have durably committed the write locally but without being gated on remote replicas to acknowledge the writes.

PS: The replication protocol of Azure Cosmos DB operates under the assumption that the read and write quorums are always local to the region where the request has been issued.

How to turn on – Cosmos Db and multi-region replication?

In CosmosDb instance settings select Replicate data globally page, then select the regions to add or remove by clicking regions in the map.

Azure Cosmos DB enables you to configure the regions (associated with the database) for “read”, “write” or “read/write” regions.

image

image 

image

Then configure Manual/Automatic failover options as well. image I would cover this in later articles.

All that said, you are in good hands of Azure Platform as a  Cosmos Db customer or user. 

NB: For the purpose of this article, I have configured my instance to run different regions with write region as East US and read region as West Europe,North Europe and West US.

image

Programatically Connect to a preferred location using the SQL API:

Now coming to the context of this blog, as a application developer some times you would like to programatically control the access to these regions while using Cosmos Db .NET SQL API. 

In CosmosDb.NET SDK version 1.8 and later, there is the ConnectionPolicy parameter for the DocumentClient constructor has a property called Microsoft.Azure.Documents.ConnectionPolicy.PreferredLocations

  • All reads will be sent to the first available region in the PreferredLocations list. If the request fails, the client will fail down the list to the next region, and so on.
  • SDK will automatically send all writes to the current write region.
  • SDK will only attempt to read from the regions specified in PreferredLocations.
  • For example: If you have 4 read regions defined in your cosmos Db instance and you only have 2 regions defined in PreferredLocations in connectionPolicy, requests from other two regions would never be served from SDK.

NB: The client application can verify the current write endpoint and read endpoint chosen by the SDK by checking two properties, WriteEndpoint and ReadEndpoint. **SDK version 1.8+.

Following code snippet would make it easiter to implement:

 
   //Setting read region selection preference. 
   connectionPolicy.PreferredLocations.Add(LocationNames.EastUS); // applications first preference
   connectionPolicy.PreferredLocations.Add(LocationNames.WestEurope); // applications second preference

Full Source Code: https://github.com/AzureContrib/CosmosDB-DotNet-Quickstart-Preferred-Location 

References:

Azure Cosmos DB – Connection Policy – Setting Connection Mode and Connection Protocol

May 13, 2018 .NET, Azure, CosmosDB, Microsoft, PaaS, VisualStudio, Windows, Windows Azure Development No comments , , ,

Recently I have been trying multiple ways to optimize CosmosDb SQL.NET SDK integration calls from my web application that sits within a VNET.

After carefully analyzing different options available within Cosmos Db SQL API’s have realized there are different aspects we could optimize in achieving minimal turn around time. In this article I am going to discuss about one such useful find, that is to use Cosmos Db SQL SDK connection policy to use diferent networking options to improve the latency between web application and cosmos db API calls.

Connection Policy:

Performance of an client application has important implication based on – how SQL .NET SDK  connects to Azure Cosmos DB , because of expected client-side latency due to networking conditions. There are two key configuration settings available for configuring client Connection Policy – the connection mode and the connection protocol.

There are two connection mode options provides by Cosmos Db SQL.NET SDK:

  • Gateway Mode(which is default): This mode is the default option being used and works with all Cosmos DB SDK versions.  Since it is only accessible over HTTPS/TCP, it is more secure and best choice for applications that run on a constrained secure corporate network. If you are using the .NET Framework version of the CosmosDb SQL.NET SDK, then proably this is the only connection mode that would work for you. 

  • Connection Protocol – TCP:  443 is the CosmosDb port, 10255 is the MongoDB API port.   
  • Connection Protocol – HTTPS: Default 443
  • Direct Mode:  This is a new mode which will work only on .NET Standard 2.0 onwards. It provides you an ability to choose between TCP or HTTPS more efficiently.  Only caveat is that you would need .NET Standard 2.0 as target framework for your client application.
    • Connection Protocol – TCP: TCP would be more faster when client and db are in same VNET.  Since TCP within the same network would be more faster, you would be amazed by the latency improvements by your client application. It would respond faster to you cosmos Db requests.  NB In TCP mode apart from 443 and 10255 mentioned in Gateway more, we also need to ensure  port range between 10000 and 20000 is open in your firewall configuration,  because Azure Cosmos DB uses dynamic TCP ports.
    • Connection Protocol – HTTPS: Since client application and cosmosDb are in same network limits, you could see that HTTPS option is also a reliable, secure and faster access channel for you, but not highly performing as TCP.

    A simplified diagram below :

    image

    Sample Code:

     string cosmosDbEndpoint = new Uri("https://mycosmosDbinstance.documents.net");
     string authKey ="cosmosDb-apiKey";
     DocumentClient client = new DocumentClient(cosmosDbEndpoint, authKey,
     new ConnectionPolicy
     {
        ConnectionMode = ConnectionMode.Direct,
        ConnectionProtocol = Protocol.Tcp
     });
     

    Refer more :

    You can find the completed sample here: AzureContrib/CosmosDB-DotNet-Quickstart-With-ConnectionPolicy

    Azure Cosmos DB name changes

    April 17, 2018 Azure, CosmosDB, Document DB, Emerging Technologies, Microsoft, Windows Azure Development No comments

    An update from Microsoft Azure says that – As part of the transition from Azure DocumentDB to Azure Cosmos DB, the service and resource names are changing from “Azure DocumentDB” to “Azure Cosmos DB” on June 1, 2018.

    How does that Impact?

    When Microsoft introduced Cosmos DB, then have ensured that there was a smooth transition or migration of existing Document DB customers /tenants to Cosmos DB. This was achieved by without changing underlying service and resource names from “Document DB” to “Cosmos DB”.

    So, if you were an existing customer of Document DB, you have noticed the only disappearance of Document DB name and old service showing simply Cosmos DB. You did not feel much difference apart from some additional configuration options as part of multi-modal data source configuration.

    Your ARM deployment templates might need some changes in resource sizing, resource location, and some other configuration aspects.

    There is no a pricing impact because of this change, but you will have to modify billing parameters that rely on the new names. Now with this deadline what Microsoft intends to have is to deprecate the use of Old DocumentDB naming and start migrating all customers/tenants to follow the new naming for the resource billing/sizing purposes.

    To read more about the naming changes: https://azure.microsoft.com/en-us/updates/name-changes-cosmos-db/

    Azure Functions App–Run OnDemand Serverless code – a path way to Serverless Computing

    June 18, 2017 App Service, Azure, Azure Functions, CosmosDB, Microsoft, Resilliancy, Scalability, Windows Azure Development, Windowz Azure No comments

    Azure Functions is a new cloud solution from Azure that would let you execute small pieces code or “functions” in the cloud.  This means you do not have to worry about the infrastructure or environment to execute your little piece of code to solve any of your business problems.

    functions-logo

    Functions can make development even more productive, and you can use your development language of choice.

    Benefits:

    • Pay only for the time your code runs and trust Azure to scale as needed.
    • Azure Functions lets you develop serveries applications on Microsoft Azure.
    • Supports wide variety of development language choices , such as C#, F#, Node.js, Python or PHP.
    • Bring your own dependencies – you can bring any of your Nuget/NPM dependencies for your functional logic.

    What can we do with Azure Functions?

    Azure Functions is a very good  solution for processing data, integrating systems, working with the internet-of-things (IoT), and building simple APIs and micro services.

    Functions provides templates to help you  get started with some useful scenarios, including the following:

    • BlobTrigger – Process Azure Storage blobs when they are added to containers. You might use this function for image resizing.
    • EventHubTrigger – Respond to events delivered to an Azure Event Hub. Particularly useful in application instrumentation, user experience or workflow processing, and Internet of Things (IoT) scenarios.
    • Generic Webhook – Process webhook HTTP requests from any service that supports webhooks.
    • GitHub Webhook – Respond to events that occur in your GitHub repositories.
    • HTTPTrigger – Trigger the execution of your code by using an HTTP request.
    • QueueTrigger – Respond to messages as they arrive in an Azure Storage queue.
    • ServiceBusQueueTrigger – Connect your code to other Azure services or on-premises services by listening to message queues.
    • ServiceBusTopicTrigger – Connect your code to other Azure services or on-premises services by subscribing to topics.
    • TimerTrigger – Execute cleanup or other batch tasks on a predefined schedule.

    Integration Support with other Azure Services:

    Following are the services integration supported by Azure Functions app.

    • Azure Cosmos DB
    • Azure Event Hubs
    • Azure Mobile Apps (tables)
    • Azure Notification Hubs
    • Azure Service Bus (queues and topics)
    • Azure Storage (blob, queues, and tables)
    • GitHub (webhooks)
    • On-premises (using Service Bus)
    • Twilio (SMS messages)

    Costing:

    Azure functions will be charged based on two pricing plans below:

    1. App Service Plan – if you already have an Azure App Service running with Logic, Web, Mobile or Web Job, you can use the same environment for your Azure functions execution without needing to pay for extra resources.  You will be charged based on regular app service rates.
    2. Consumption plan  – with this plan you only need to pay for how long and how many times your functions runs and computational needs/resource usage during that execution time. Consumption plan pricing includes a monthly free grant of 1 million requests and 400,000 GB-s of resource consumption per month.

    You can find further pricing related info here

    Support and SLA:

    • Free billing and subscription management support
    • Flexible support plans starting at $29/month. Find a plan
    • 99.95% guaranteed up time. Read the SLA

    Useful Links: