APIError: Never received confirmation dataset was published

Using pyrasgo
, , ,
1

I am running a

rasgo.publish.dataset( )

to an existing resource_key and I’m getting this error:

---------------------------------------------------------------------------
APIError                                  Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_19256/740188593.py in <module>
----> 1 rasgo.publish.dataset(dataset=hist_history_input, 
      2                       name= "Hist Snapshot table - pre 2019",
      3                       resource_key=outname_snapshot,
      4                       table_type='TABLE',
      5                       description="Hist Snapshot table - pre 2019")

~\Anaconda3\lib\site-packages\pyrasgo\api\publish.py in dataset(self, dataset, name, resource_key, description, verbose, time_index, attributes, table_type, table_name, generate_stats, timeout)
     86                 attributes = {"time_index": time_index}
     87 
---> 88         dataset = self.create._dataset(
     89             name=name,
     90             source_type=DatasetSourceType.RASGO,

~\Anaconda3\lib\site-packages\pyrasgo\api\create.py in _dataset(self, name, source_type, resource_key, description, status, dw_table_id, dw_operation_set_id, fqtn, attributes, publish_ds_table_type, publish_ds_table_name, generate_stats, timeout)
    252                 raise APIError("Timeout reached waiting for dataset creation.")
    253             time.sleep(POLL_RETRY_RATE)
--> 254         raise APIError(f"Never received confirmation dataset was published.")
    255 
    256     def _operation_set_non_async(

APIError: Never received confirmation dataset was published.
2

This error occurs when pyrasgo triggers a dataset to be created asynchronously, and then pyrasgo times out polling the api for completion status. This is most likely due to Snowflake taking a very long time to publish the data as a table_type=‘TABLE’

Please check the dataset with rasgo.get.dataset(resource_key=resource_key) to inspect whether a new version is being returned.

3

This topic was automatically closed after 365 days. New replies are no longer allowed.