Aggregator not emmiting UsageSubmittedToAPI event due to missing headers

[luchocarbonell]

Resources context

I deployed an event hub and a storage account by myself in the Azure portal. Also, I created a SaaS application managed by me and my team to test the billing accelerator.

I'm even using the ReplayCaptureForPartition solution (as shown in this tutorial) to see the events we process. If I don't apply any workaround, I'm never going to get a UsageSubmittedToAPI event back, so users wouldn't get charged.

Issue description

When the Aggregator receives the response from the Marketplace API (batchUsageEvent), the method submitBatchUsage tries to get the values for the headers x-ms-correlationid and x-ms-requestid, but those values don't exist in the response, even when accepted.

Exception when the code tries to find those headers

Incoming headers in the response

The line where the error is thrown is line 33 in the metered-billing-accelerator\src\Metering.Runtime\MarketplaceClient.fs file

I'm I missing something?

Proposed workarounds

I'm not really fluent in F#, but I tested 2 workarounds:

1. Check if the headers exist. If they exist, we use their values, otherwise, we generate their values:

let submitBatchUsage (config: MeteringConfigurationProvider) (usage: MarketplaceRequest list) : Task<MarketplaceBatchResponse> =
    task {
        let! client = InstanceMetadataClient.createMarketplaceClient config.MeteringConnections.MeteringAPICredentials
        let json = usage |> MarketplaceBatchRequest.createBatch |> Json.toStr 0

        let request = new HttpRequestMessage(
            method = HttpMethod.Post,
            requestUri = $"/api/batchUsageEvent?api-version={meteringApiVersion}",
            Content = new StringContent(json, Encoding.UTF8, "application/json"))

        let! response = client.SendAsync(request)
        let! responseJson = response.Content.ReadAsStringAsync()

        let header name =
            match response.Headers.TryGetValues(name) with
            | true, values -> String.concat " " values
            | false, _ -> Guid.NewGuid().ToString()

        let azureHeader = { RequestID = header "x-ms-requestid"; CorrelationID = header "x-ms-correlationid" }

        return
            responseJson
            |> Json.fromStr<MarketplaceBatchResponseDTO>
            |> (fun x -> x.Results)
            |> List.map (MarketplaceResponse.create azureHeader)
            |> MarketplaceBatchResponse.create
    }

Capture file screenshot from workaround number 1

2. Delete the line and set fixed values:

 let submitBatchUsage (config: MeteringConfigurationProvider) (usage: MarketplaceRequest list) : Task<MarketplaceBatchResponse> =
     task {
         let! client = InstanceMetadataClient.createMarketplaceClient config.MeteringConnections.MeteringAPICredentials
         let json = usage |> MarketplaceBatchRequest.createBatch |> Json.toStr 0

         let request = new HttpRequestMessage(
             method = HttpMethod.Post,
             requestUri = $"/api/batchUsageEvent?api-version={meteringApiVersion}",
             Content = new StringContent(json, Encoding.UTF8, "application/json"))

         let! response = client.SendAsync(request)
         let! responseJson = response.Content.ReadAsStringAsync()

         let azureHeader = { RequestID = "x-ms-requestid"; CorrelationID = "x-ms-correlationid" }

         return
             responseJson
             |> Json.fromStr<MarketplaceBatchResponseDTO>
             |> (fun x -> x.Results)
             |> List.map (MarketplaceResponse.create azureHeader)
             |> MarketplaceBatchResponse.create
     }

Capture file screenshot from workaround number 2