diff --git a/docs/components/components.md b/docs/components/components.md index d2bf2d70..2c5d3936 100644 --- a/docs/components/components.md +++ b/docs/components/components.md @@ -1,5 +1,3 @@ -from distributed import Client - # Components Fondant makes it easy to build dataset collaborative leveraging reusable components. Fondant @@ -70,17 +68,18 @@ schema defined by the `produces` section of the component specification. ### Configuring Dask You can configure the [Dask client](https://docs.dask.org/en/stable/scheduling.html) based on the -needs of your component by overriding the `dask_client` method: +needs of your component by overriding the `setup` method: ```python import os +import typing as t from dask.distributed import Client, LocalCluster from fondant.component import PandasTransformComponent class Component(PandasTransformComponent): - def dask_client(self) -> Client: + def setup(self) -> t.Any: """Initialize the dask client to use for this component.""" cluster = LocalCluster( processes=True, diff --git a/docs/dataset.md b/docs/dataset.md index e0fcab90..1ffa335f 100644 --- a/docs/dataset.md +++ b/docs/dataset.md @@ -13,7 +13,7 @@ You can initialise a dataset from a previous run by using the `read` method. ```python from fondant.dataset import Dataset -dataset = Dataset.read("path/to/manfiest.json") +dataset = Dataset.read("path/to/manifest.json") ``` diff --git a/docs/guides/build_a_simple_dataset.md b/docs/guides/build_a_simple_dataset.md index d4bd4180..62d1084c 100644 --- a/docs/guides/build_a_simple_dataset.md +++ b/docs/guides/build_a_simple_dataset.md @@ -99,7 +99,7 @@ We provide three arguments to the `.create()` method: To materialize your dataset, you can execute the following command within the directory: ```bash -fondant run local dataset.py --working_direcotry ./data +fondant run local dataset.py --working-directory ./data ``` The workflow execution will start, initiating the download of the dataset from HuggingFace. diff --git a/docs/runners/local.md b/docs/runners/local.md index 437b21db..e07100d4 100644 --- a/docs/runners/local.md +++ b/docs/runners/local.md @@ -17,7 +17,7 @@ about this in the [installation](../guides/installation.md) guide. ### Running a dataset with the local runner -Fondant will create a default working directory (for intermediate artifacts) for the dataset in the current working directory called '/.artifacts'. You can override this by passing the `--working-dir` argument to the `run` command. Or by setting the `working_dir` argument in the `run` method of the `DockerRunner` class. +Fondant will create a default working directory (for intermediate artifacts) for the dataset in the current working directory called '/.artifacts'. You can override this by passing the `--working-directory` argument to the `run` command. Or by setting the `working_directory` argument in the `run` method of the `DockerRunner` class. === "Console" diff --git a/docs/runners/sagemaker.md b/docs/runners/sagemaker.md index d5f23c20..a732cca6 100644 --- a/docs/runners/sagemaker.md +++ b/docs/runners/sagemaker.md @@ -49,7 +49,7 @@ AWS with a role that has all the required permissions to launch a SageMaker pipe ```bash fondant run sagemaker \ - --working-dir $S3_BUCKET \ + --working-directory $S3_BUCKET \ --role-arn $SAGEMAKER_ROLE_ARN ``` @@ -61,8 +61,8 @@ AWS with a role that has all the required permissions to launch a SageMaker pipe runner = SageMakerRunner() runner.run( - input=, - working_dir=, + dataset=, + working_directory=, role_arn=, pipeline_name= ) diff --git a/docs/runners/vertex.md b/docs/runners/vertex.md index 1f65aab7..d16fe1fc 100644 --- a/docs/runners/vertex.md +++ b/docs/runners/vertex.md @@ -56,7 +56,7 @@ info [here](https://codelabs.developers.google.com/vertex-pipelines-intro#2) region=project_region, service_account=service_account) ) - runner.run(input_spec=, working_dir=) + runner.run(dataset=, working_directory=) ```