Cloudflare has a great Analytics dashboard that provides good insight into your proxied web traffic. While the interface is intuitive and great for some high-level information, the dashboard may not always have the information you need.
The advanced logs you are looking for can be retrieved using either Logpush or Logpull. In this post we will cover the basics and retrieve some logs using the Logpull API with PowerShell.
Gather Requirements
In order to make requests, you will need an API key and a zone ID.
Log in to Cloudflare and go to My Profile:
Go to API Tokens and then view your Global API Key. Save this for later:
Navigate to your Account and then choose your website you want to view logs for. Retrieve the Zone ID and save this for later:
Enable Log Retention
HTTP request logs are not retained by default, so you will need to enable log retention before you can use the Logpull API.
You can check the status of log retention for your zone by running this in PowerShell (replacing variables with the values you retrieved earlier):
#Cloudflare API Key
$apiKey = "YOUR_API_KEY_HERE"
#Cloudflare Zone
$zoneId = "YOUR_ZONE_ID_HERE"
#Cloudflare Profile Email Address
$email = "YOUR_EMAIL_ADDRESS_HERE"
#Request Endpoint
$url = "https://api.cloudflare.com/client/v4/zones/$zoneId/logs/control/retention/flag"
#Required authentication headers
$headers = @{
"X-Auth-Key" = $apiKey
"X-Auth-Email" = $email
"Content-Type" = "application/json"
}
#Send request, store response
$response = Invoke-RestMethod -Method Get -Uri $url -Headers $headers
#Print response
$responseIf the flag in the response is set to false, you need to enable log retention. Run the following to enable log retention:
#Cloudflare API Key
$apiKey = "YOUR_API_KEY_HERE"
#Cloudflare Zone
$zoneId = "YOUR_ZONE_ID_HERE"
#Cloudflare Profile Email Address
$email = "YOUR_EMAIL_ADDRESS_HERE"
#Request Endpoint
$url = "https://api.cloudflare.com/client/v4/zones/$zoneId/logs/control/retention/flag"
#Required authentication headers
$headers = @{
"X-Auth-Key" = $apiKey
"X-Auth-Email" = $email
"Content-Type" = "application/json"
}
#Data payload
$body = @{
"flag" = $true
} | ConvertTo-Json
#Send request, store response
$response = Invoke-RestMethod -Method Post -Uri $url -Headers $headers -Body $body
#Print response
$responseMaking a Request
With log retention enabled, you are ready to start making requests. You will want to review the available log fields to customize your request to conform to your use case.
Here is an example of a request made to the /logs/received endpoint which will fetch the previous hour's worth of logs for your zone and save them to a .ndjson file locally:
#Specify filepath to export logs
$exportPath = "C:\some\file\path.ndjson"
#Cloudflare API Key
$apiKey = "YOUR_API_KEY_HERE"
#Cloudflare Zone
$zoneId = "YOUR_ZONE_ID_HERE"
#Cloudflare Profile Email Address
$email = "YOUR_EMAIL_ADDRESS_HERE"
#Get time from 1 hour ago in ISO 8601 format; e.g. 2022-03-23T02:30:19Z
$timeStart = (Get-Date -AsUTC).AddHours(-1).ToString("yyyy-MM-ddTHH:mm:ss") + "Z"
#Get time from 1 minute ago in ISO 8601 format; e.g. 2022-03-23T03:29:19Z
$timeEnd = (Get-Date -AsUTC).AddMinutes(-1).ToString("yyyy-MM-ddTHH:mm:ss") + "Z"
#Specify fields to return
$fields = "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID"
#Request endpoint with parameters
$url = "https://api.cloudflare.com/client/v4/zones/$zoneId/logs/received?start=$timeStart&end=$timeEnd&fields=$fields"
#Required authentication headers
$headers = @{
"X-Auth-Key" = $apiKey
"X-Auth-Email" = $email
"Content-Type" = "application/json"
}
#Send request, store response
$response = Invoke-RestMethod -Method Get -Uri $url -Headers $headers
#Create .ndjson file at $exportPath if it doesn't already exist, add contents of response to file
if (!(Test-Path -Path $exportPath)) {
New-Item -Path $exportPath
Add-Content -Path $exportPath -Value $response
} else {
Add-Content -Path $exportPath -Value $response
}View the Cloudflare developer documentation for more information about requesting logs with the Logpull API, which includes all of the available log fields that are supported to customize your requests.
