Hi, thanks for reaching out. A few fixes to how you invoke your query should fix the problem:
- If you have a LIMIT like 10000, the query runs fast enough for sync mode.
- Note also that to get a pandas dataframe, the argument is 'fmt' (not 'format').
- If you wish to write out the result to a CSV file on vospace, I think you must also use fmt='csv' (which is the default by the way).
- If you are asking for a pandas dataframe as the returned result, you don't do a manual .convert() afterwards, since you already have a dataframe.
- Finally, your query string is SQL, not ADQL (in ADQL you'd use 'SELECT TOP 10000 ...' instead of '... LIMIT 10000' for instance).
So, these all work for me:
df = qc.query(sql=aquery, fmt='pandas') # with LIMIT 10000 takes 0.4s
Or with manual conversion:
res = qc.query(sql=aquery, fmt='csv')
df = convert(res, outfmt='pandas')
If you wanted to write out a CSV file:
res = qc.query(sql=aquery, fmt='csv', out='vos://my_file1.csv')
which you can then retrieve and convert at any time with:
from dl import storeClient as sc
data = sc.get('vos://my_file1.csv')
df = convert(data)
Finally, for longer runs, the async version, writing out to a CSV file on vospace (plus a trick to do the async-wait-loop automatically):
res = qc.query(sql=aquery, fmt='csv', async_=True, wait=True, verbose=True, poll=10, out='vos://my_file2.csv')
df = convert(res)
Hope this is helpful. Let us know if you encounter more problems.
Robert