# ?take=all Paramater

**URL:** <https://forum.api.combocurve.com/t/take-all-paramater/65>\
**Category:** Guidance\
**Tags:** python\
**Created:** [December 29, 2021, 7:23pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65 "2021-12-29T19:23:35Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 29, 2021, 7:23pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/1 "2021-12-29T19:23:36Z")

</div>

Question - how do I set my take on GET requests to all?

```auto
urltwo = (
    "https://api.combocurve.com/v1/projects/61a92c8f34254c0013cacf3e/scenarios/61a93338b763c20015f3f68f/econ-runs/"
    + econId
    + "/monthly-exports/"
    + econRunId
    + "?skip=0"
    + "&take=200"
)

```

right now I am just using a number I know is bigger than my rows, but I want to see if there is a way to set it to grab all the values.

Thanks!

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 29, 2021, 7:44pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/2 "2021-12-29T19:44:16Z")

</div>

@michael.tanner good question.

There’s a maximum allowed value for every `take` parameter. On this endpoint in particular it’s `200`, like you have in your snippet. This is mainly because there’s only so much data is recommended to transfer in a single HTTP request + response.

So the expected is that caller do pagination. Which means repeating the request with a fixed `take`, but incrementing `skip` until you get no records back.

Pseudo-code would be roughly something like:

```python
MAX_TAKE = 200

skip = 0
has_more = True

while has_more:
  response = make_request(skip, MAX_TAKE)
  process_response(response)
  skip += len(response)
  has_more = len(response) == MAX_TAKE # if len(response) < take then this is the last page

```

Hopefully we will add helpers for this in Python in the near future, like the ones we have in the C# client to do this pagination work for you.

I hope that helps.

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 29, 2021, 7:50pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/3 "2021-12-29T19:50:06Z")

</div>

@maikel.ruiz thanks! Wow I got lucky with picking 200.

Will add this to my code here - working on summing up a few different output rows (all the taxes) into one column.

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 29, 2021, 7:51pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/4 "2021-12-29T19:51:49Z")

</div>

Haha, got it.

We have a guide for this topic that will be coming in the near future with more explanation:

> [@Pagination](https://forum.api.combocurve.com/t/pagination/33/2):
>
> Came up on this question: [?take=all Paramater](https://forum.api.combocurve.com/t/take-all-paramater/65)

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 29, 2021, 9:58pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/5 "2021-12-29T21:58:31Z")

</div>

@michael.tanner check the latest version (`0.2.3`) [combocurve-api-v1 · PyPI](https://pypi.org/project/combocurve-api-v1/)

We just added the `get_next_page_url` helper. Example:

```python
from combocurve_api_v1.pagination import get_next_page_url

# See Authorization section
auth_headers = combocurve_auth.get_auth_headers()

# Additional filters are allowed, it is preferred not specify skip if its value is 0
url = 'https://api.combocurve.com/v1/wells?take=200'

# First request
has_more = True

# Keep fetching while there are more records to be returned
while has_more:
    response = requests.get(url, headers=headers)

    # Process response

    url = get_next_page_url(response.headers)
    has_more = url is not None

```

On the surface this mainly manages the `skip` parameter for you. But this is in fact more robust and will also transparently allow other more optimized pagination methods in the future.

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 29, 2021, 10:13pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/6 "2021-12-29T22:13:49Z")

</div>

just pip upgraded to new version, worked great

this is exactly what I need to finish this project - you team rocks per usual

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 29, 2021, 10:28pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/7 "2021-12-29T22:28:36Z")

</div>

Glad to hear, and thanks. We appreciate your collaboration to improve our tooling and documentation.

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 29, 2021, 11:05pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/8 "2021-12-29T23:05:39Z")

</div>

Ok little help here. The below code is my first attempt…it never exits the while loop so I added the print statement…not sure how to say has\_more go to false?

```auto
# Reautenticated client
auth_headers = combocurve_auth.get_auth_headers()
# set new url with econRunID, skipping zero
urltwo = (
    "https://api.combocurve.com/v1/projects/61a92c8f34254c0013cacf3e/scenarios/61a93338b763c20015f3f68f/econ-runs/"
    + econId
    + "/monthly-exports/"
    + econRunId
    + "?take=200"
)

has_more = True

while has_more:
    # same as above, parsing as JSON string
    response = requests.request("GET", urltwo, headers=auth_headers)
    urltwo = get_next_page_url(response.headers)
    has_more = urltwo is not None
    print(len(response.text))

```

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 29, 2021, 11:38pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/9 "2021-12-29T23:38:38Z")

</div>

That’s weird. I have this working code, I only removed the authentication part, and it’s not very different from yours:

```python
import requests
from combocurve_api_v1.pagination import get_next_page_url

'''
    TODO: Authentication steps: instantiatie `combocurve_auth`
'''

BASE_URL = 'https://api.combocurve.com'

url = f'{BASE_URL}/v1/projects?take=200'

has_more = True

# Keep fetching while there are more records to be returned
while has_more:
    print(url) # see the URL for each request
    headers = combocurve_auth.get_auth_headers()
    response = requests.get(url, headers=headers)

    # Process response here
    # print(len(response.json()))

    url = get_next_page_url(response.headers)
    has_more = url is not None

```

Maybe it takes a while inside the while because it’s actually making more requests.  
You can also give it a try in a different resource, with fewer records, like in this example using `/projects`.

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 29, 2021, 11:45pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/10 "2021-12-29T23:45:14Z")

</div>

@maikel.ruiz hmmm thanks for the sample code! Yeah - its only a 4 well test but for the actual cash flow run I am pulling will have 130 wells.

I wonder, letting it run for a little see if it pops at some point.

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 30, 2021, 12:18am UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/11 "2021-12-30T00:18:20Z")

</div>

I noticed earlier I tried with one-liner. Now when I tried with the cash flows It’s indeed looping forever so you may have found a bug in the API for this particular endpoint. Looking into this on our side.

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 30, 2021, 2:20am UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/12 "2021-12-30T02:20:39Z")

</div>

@maikel.ruiz you rock!! I am seeing if there is anything I can do on my side as well

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 30, 2021, 9:37pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/13 "2021-12-30T21:37:20Z")

</div>

@michael.tanner we deployed earlier the fix we were working on for the cashflows pagination and I just verified it is working as expected now.

Thanks for you patience 🙂

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 30, 2021, 9:50pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/14 "2021-12-30T21:50:03Z")

</div>

@maikel.ruiz nice!! Yes…it works!! Pops out of the loop

Here is my issue…how do I store the data from each of the URL’s…

```auto
urltwo = (
    "https://api.combocurve.com/v1/projects/61a92c8f34254c0013cacf3e/scenarios/61a93338b763c20015f3f68f/econ-runs/"
    + econId
    + "/monthly-exports/"
    + econRunId
    + "?take=200"
)

has_more = True

while has_more:
    response = requests.request("GET", urltwo, headers=auth_headers)
    urltwo = get_next_page_url(response.headers)
    has_more = urltwo is not None

jsonStr = response.text
print(len(jsonStr))
# print(jsonStr)
dataObj = json.loads(jsonStr)

dataResults = dataObj["results"]

```

dataResults is length 43, which is great, should have 243 records…but I am missing the part where I save the first 200 to a table.

Thanks for getting on this so quickly!! I was working through pagination youtube and had something simlinar to your psuedo code but it was much bulkier and still struggled with the saving each request

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 30, 2021, 10:08pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/15 "2021-12-30T22:08:53Z")

</div>

Yeah, expected, you have the `print` outside the `while`, so you are only printing for the last value `response` had, and not for every response.

Something like this will print the length for every page:

```python
urltwo = (
    "https://api.combocurve.com/v1/projects/61a92c8f34254c0013cacf3e/scenarios/61a93338b763c20015f3f68f/econ-runs/"
    + econId
    + "/monthly-exports/"
    + econRunId
    + "?take=200"
)

has_more = True

while has_more:
    response = requests.request("GET", urltwo, headers=auth_headers)
    urltwo = get_next_page_url(response.headers)
    process_page(response.json())
    has_more = urltwo is not None

def process_page(response_json):
    results = response_json["results"]

    # TODO: process `results`, write to DB or append to total result
    print(len(results))

```

Also note how response.json() saves a few steps.

---

<div class="post-metadata">

**Author:** ![michael.tanner](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/michael.tanner/32/34_2.png) [@michael.tanner](https://forum.api.combocurve.com/u/michael.tanner)\
**Post date:** [December 30, 2021, 10:15pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/16 "2021-12-30T22:15:39Z")

</div>

Ah ok make sense! Thanks - will work this solution here. @maikel.ruiz

---

<div class="post-metadata">

**Author:** ![maikel.ruiz](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.api.combocurve.com/maikel.ruiz/32/101_2.png) [@maikel.ruiz](https://forum.api.combocurve.com/u/maikel.ruiz)\
**Post date:** [December 31, 2021, 6:02pm UTC](https://forum.api.combocurve.com/t/take-all-paramater/65/17 "2021-12-31T18:02:54Z")

</div>