Getting started with pulling monthly forecast volumes

Guidance
, ,
1

I am trying to pull my monthly forecast volumes from a particular forecast and had a couple of questions:

  1. Does anyone have a script or function that turns the json format that gets returned into a nice dataframe format? i.e. wellid, projectid, date, phase, value
  2. Why does the row count that gets returned from this query not equal (well count in project) * (phases)? I am querying a project that has ~200 wells, but only 66 records are returned from this query. I originally thought it was pagination, but updated to use the pagination method as documented
  3. How should I take ‘well’ and connect it to other attributes of a well (i.e. well name, api10, etc). Should i take the id’s that get returned and run get-well-by-id?
import requests
import pandas as pd
from cc_helper import get_cc_auth
from combocurve_api_v1.pagination import get_next_page_url


# 1. Credentials & IDs
bearer_token = get_cc_auth()
api_key = ""

project_id = ""  # From your URL
forecast_id = "" # From your URL

# 2. Construct the Scoped URL
url = f"https://api.combocurve.com/v1/forecast-monthly-volumes"

# 3. Correct Header Formatting
headers = {
    'Accept': 'application/json',
    'x-api-key': api_key,
    'Authorization': bearer_token # Note the "Bearer " prefix
}

payload = {
    'forecast': forecast_id,
    'project': project_id
}

def extract_paths(obj, prefix=""):
    paths = []

    if isinstance(obj, dict):
        for k, v in obj.items():
            new_prefix = f"{prefix}.{k}" if prefix else k
            paths.append(new_prefix)
            paths.extend(extract_paths(v, new_prefix))

    elif isinstance(obj, list) and obj:
        paths.extend(extract_paths(obj[0], prefix + "[]"))

    return paths

# 3. Get the data
has_more = True

while has_more:
    response = requests.get(url, payload, headers=headers)
    url = get_next_page_url(response.headers)
    has_more = url is not None

if response.status_code == 200:
    json_data = response.json()

    # 4. Convert JSON list to a Pandas DataFrame (Table)
    if json_data:
        df = pd.json_normalize(json_data, 'phases')

    else:
        print("The forecast is empty.")
else:
    print(f"Error: {response.status_code} - {response.text}")