Description
Today, Amazon provider package supports SageMaker operators for a few operations, like training, tuning, pipelines, but it lacks the support for SageMaker Notebook instances. Boto3 provides necessary APIs to create notebook instance, start notebook instance, stop network instance and delete notebook instance. Leveraging these APIs, we should add new operators to SageMaker set under Amazon provider. At the same time, having a sensor (synchronous as well as deferrable) for notebook instance execution that utilizes describe notebook instance and waits for Stopped/Failed status would help with observability of the execution.
Use case/motivation
Data Scientists are orchestrating ML use cases via Apache Airflow. A key component of ML use cases is running Jupyter Notebook on SageMaker. Having built-in operators and sensors would make it easy for Airflow users to run Notebook instances on SageMaker.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
Description
Today, Amazon provider package supports SageMaker operators for a few operations, like training, tuning, pipelines, but it lacks the support for SageMaker Notebook instances. Boto3 provides necessary APIs to create notebook instance, start notebook instance, stop network instance and delete notebook instance. Leveraging these APIs, we should add new operators to SageMaker set under Amazon provider. At the same time, having a sensor (synchronous as well as deferrable) for notebook instance execution that utilizes describe notebook instance and waits for Stopped/Failed status would help with observability of the execution.
Use case/motivation
Data Scientists are orchestrating ML use cases via Apache Airflow. A key component of ML use cases is running Jupyter Notebook on SageMaker. Having built-in operators and sensors would make it easy for Airflow users to run Notebook instances on SageMaker.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct