# Get daily production not getting all records

**URL:** <https://forum.api.combocurve.com/t/get-daily-production-not-getting-all-records/250>\
**Category:** Guidance\
**Tags:** python\
**Created:** [October 12, 2023, 7:58pm UTC](https://forum.api.combocurve.com/t/get-daily-production-not-getting-all-records/250 "2023-10-12T19:58:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sevans\_Callon](https://avatars.discourse-cdn.com/v4/letter/s/ba8739/32.png) [@Sevans\_Callon](https://forum.api.combocurve.com/u/Sevans_Callon)\
**Post date:** [October 12, 2023, 7:58pm UTC](https://forum.api.combocurve.com/t/get-daily-production-not-getting-all-records/250/1 "2023-10-12T19:58:27Z")

</div>

Hello, I’m trying to pull down all of the daily production data from a project. Below is my script. I’m using the pagination example here ([combocurve-api-v1 · PyPI](https://pypi.org/project/combocurve-api-v1/)), with a few modifications to dump it into a pandas dataframe.

My problem is the script is only returning a fraction of the daily production. My project has ~128K records, and I’m only getting ~39K. Any suggestions on what I’m doing wrong?

```auto
from combocurve_api_v1 import ServiceAccount, ComboCurveAuth
import requests
import pandas as pd
from combocurve_api_v1.pagination import get_next_page_url

url_daily_production = f'https://api.combocurve.com/v1/projects/{projectid}/daily-productions'

url = url_daily_production + f'?take={20000}'

response_list = []

has_more = True

# Keep fetching while there are more records to be returned
while has_more:
    
    response = requests.get(url, headers=auth_headers)

    response_list = response_list + response.json()

    url = get_next_page_url(response.headers)
    has_more = url is not None

df_with_all_records = pd.DataFrame(response_list)

```

---

<div class="post-metadata">

**Author:** ![HGantz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/hgantz/32/95_2.png) [@HGantz](https://forum.api.combocurve.com/u/HGantz)\
**Post date:** [October 12, 2023, 8:33pm UTC](https://forum.api.combocurve.com/t/get-daily-production-not-getting-all-records/250/2 "2023-10-12T20:33:02Z")

</div>

Hey @Sevans_Callon welcome to the forum!  
You are correct. This is a bug that is affecting daily and monthly productions using the cursor.

Here is another post about the issue:

> [@HEAD and GET counts do not match](https://forum.api.combocurve.com/t/head-and-get-counts-do-not-match/249):
>
> HEAD monthly-production https://api.combocurve.com/v1/monthly-productions?updatedAt[ge]=2023-10-02 HEAD Returns X-Query-Count - 21670 GET monthly-productions First call https://api.combocurve.com/v1/monthly-productions?take=2000&updatedAt[ge]=2023-10-02 Returns 2K records and Next and First links . Next link is https://api.combocurve.com/v1/monthly-productions?take=2000&updatedAt%5Bge%5D=2023-10-02&cursor=NjQ1MTg2OTcwNWRmODkyN2Y2YWUwMjMwfDI%3D Next page call https://api.combocurv…

We already have a PR with a fix in place and it should be going out in the next release. In the meantime, you can use skip / take pagination for these endpoints as a workaround.

---

<div class="post-metadata">

**Author:** ![Sevans\_Callon](https://avatars.discourse-cdn.com/v4/letter/s/ba8739/32.png) [@Sevans\_Callon](https://forum.api.combocurve.com/u/Sevans_Callon)\
**Post date:** [October 15, 2023, 6:47pm UTC](https://forum.api.combocurve.com/t/get-daily-production-not-getting-all-records/250/3 "2023-10-15T18:47:04Z")

</div>

Thanks, the skip/take method worked. Attaching is my script for anyone needing a python solution for the daily production.

```auto
skip = 0
take = 20000
response_count = 1

url = f'https://api.combocurve.com/v1/projects/{projectid}/daily-productions?skip={skip}&take={take}'

response_list = []

# Keep fetching while there are more records to be returned
while response_count != 0:
    
    response = requests.get(url, headers=auth_headers)
    response_count = len(response.json())
    response_list = response_list + response.json()
    
    skip = skip + take

    url = f'https://api.combocurve.com/v1/projects/{projectid}/daily-productions?take={take}&skip={skip}'

df_with_all_records = pd.DataFrame(response_list)

```

And just an FYI. I was using the daily production csv export in CC to test if I was getting an accurate number of records back. For some reason the API was returning a few hundred records short. It turns out I imported some production data that did not match any of the wells in the wellheader table in my project. The csv export in CC returns all daily production regardless if it has a well header match, and it looks like the API only returns records that have a match in the wellheader table.

CC Team, please correct me if I have that wrong.

---

<div class="post-metadata">

**Author:** ![HGantz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/hgantz/32/95_2.png) [@HGantz](https://forum.api.combocurve.com/u/HGantz)\
**Post date:** [October 26, 2023, 11:41pm UTC](https://forum.api.combocurve.com/t/get-daily-production-not-getting-all-records/250/4 "2023-10-26T23:41:27Z")

</div>

Hey, I just wanted to reply and let you know that a fix for this issue has been released. You should now be able to use cursor pagination on daily and monthly productions.