Repository navigation
Handling ConnectionError, HTTPError thrown during a query #130
Description
Versions
OS: Linux (Ubuntu 18.04)
Python: 3.8.6
krakenex: 2.1.0
What are you trying to achieve?
I'm looking to record price data in a persistent session. I do this through a simple wrapper class
class KrakenSession(object):
def __init__(self):
self.k = krakenex.API()
self.k.load_key('src/kraken.key')
def get_current_asking_price(self):
"""queries kraken API for currency pair
:note: don't query this too fast or you'll get request errors
0.25 seconds per request or faster is too fast
"""
pairstr='XXBTZUSD'
query = self.k.query_public('Ticker', {'pair':pairstr})
asking_price = query['result'][pairstr]['a'][0]What happens instead?
The above snippet works for a few hours, then inevitably crashes
Traceback (most recent call last):
File "src/kraken.py", line 96, in <module>
krs.record()
File "src/kraken.py", line 84, in record
price = self.get_current_asking_price(pairstr='XXBTZUSD')
File "src/kraken.py", line 31, in get_current_asking_price
query = self.k.query_public('Ticker', {'pair':pairstr})
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/krakenex/api.py", line 162, in query_public
return self._query(urlpath, data, timeout = timeout)
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/krakenex/api.py", line 134, in _query
self.response = self.session.post(url, data = data, headers = headers,
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/requests/sessions.py", line 590, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))I presume this could be a problem on the Kraken end (or perhaps with my internet connection), so I wrap the body of get_current_asking_price in a try/except:
def get_current_asking_price(self):
"""queries kraken API for currency pair
:note: don't query this too fast or you'll get request errors
0.25 seconds per request or faster is too fast
"""
pairstr='XXBTZUSD'
try:
query = self.k.query_public('Ticker', {'pair':pairstr})
asking_price = query['result']['XXBTZUSD']['a'][0]
except ConnectionError:
print('encountered a server error. retrying...')
time.sleep(60)
query = self.k.query_public('Ticker', {'pair':pairstr})
asking_price = query['result'][pairstr]['a'][0]
return float(asking_price)which again runs for a few hours, but returns a different error (without printing 'encountered a server error. retrying...'):
Traceback (most recent call last):
File "src/kraken.py", line 102, in <module>
krs.record()
File "src/kraken.py", line 90, in record
price = self.get_current_asking_price(pairstr='XXBTZUSD')
File "src/kraken.py", line 32, in get_current_asking_price
query = self.k.query_public('Ticker', {'pair':pairstr})
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/krakenex/api.py", line 162, in query_public
return self._query(urlpath, data, timeout = timeout)
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/krakenex/api.py", line 138, in _query
self.response.raise_for_status()
File "/home/foucartc/anaconda3/envs/ctrade_env/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 520 Server Error: for url: https://api.kraken.com/0/public/Tickerwhich seems like a deeper error in requests. Before I make my try/except handling more elaborate, I want to understand what's going on here.
- Is this a problem on the kraken side?
- Does it require resetting / opening a new connection through krakenex and closing the current session?
- Something else?
Any pointers would be appreciated!
Activity
veox commentedon Feb 28, 2021
More often than not. You can avoid a few
'Connection reset by peer'by sending less queries. However, during a DDoS or heavy load Kraken may choose to drop/reset the connection regardless.It can also happen due to a flaky connection.
See the approaches in PRs #99 and #100, as well as the issues linked there, for a three-year old (:disappointed:) discussion on this.
I never got to merging either approach into the package, since both solutions have issues, and are hard to test properly these days.
Github says either of those can still be merged into the code as-is.
Be careful with retries:
https://www.reddit.com/r/krakenex/comments/778uvh/psa_http_error_502_does_not_mean_the_query_wont/
This is old, and Kraken has had an iteration on their production code, so the PSA may not apply verbatim - just linking as an example of "unexpected things can happen".
veox commentedon Feb 28, 2021
Marking as duplicate of issue #66, mainly because the solution seems to be the same. Ping back if I misunderstood.
CorbinFoucart commentedon Mar 26, 2021
For those who encounter the same problem, I found the solutions in PRs #99 and #100 were insufficient to prevent the program from crashing. My approach was instead to wrap the price querying code in a shell script that restarts upon failure, which seems to work great although it doesn't solve the underlying problem at the krakenex level.
Closing. Thank you for the input!