Skip to content

[SPARK-47374][CONNECT][DOCS] Fix connect-repl usage prompt & docs link #45494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
panbingkun wants to merge 2 commits into from
Closed

[SPARK-47374][CONNECT][DOCS] Fix connect-repl usage prompt & docs link #45494

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
89aea6a
[SPARK-47374][CONNECT][DOCS] Fix connect-repl `usage prompt` & `docs …
panbingkun Mar 13, 2024
eddc452
fix code style
panbingkun Mar 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .../common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClientParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import scala.annotation.tailrec
*/
private[sql] object SparkConnectClientParser {

// scalastyle:off line.size.limit
/**
* @return
* usage string.
Expand All @@ -34,12 +35,15 @@ private[sql] object SparkConnectClientParser {
| --remote REMOTE URI of the Spark Connect Server to connect to.
| --host HOST Host where the Spark Connect Server is running.
| --port PORT Port where the Spark Connect Server is running.
| --enable-ssl Connect to the server using SSL.
| --use_ssl Connect to the server using SSL.

@panbingkun panbingkun Mar 13, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously, it is use_ssl here
https://github.com/apache/spark/blob/master/connector/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClientParser.scala#L63

| --token TOKEN Token to use for authentication.
| --user_id USER_ID Id of the user connecting.
| --user_name USER_NAME Name of the user connecting.
| --user_agent USER_AGENT The User-Agent Client information (only intended for logging purposes by the server).

@panbingkun panbingkun Mar 13, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An explanation of user_agent

spark/connector/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClient.scala

Line 676 in c7795bb

userAgent: String = genUserAgent(

spark/connector/connect/common/src/main/protobuf/spark/connect/base.proto

Lines 72 to 75 in c7795bb

// Provides optional information about the client sending the request. This field
// can be used for language or version specific information and is only intended for
// logging purposes and will not be interpreted by the server.
optional string client_type = 3;

| --session_id SESSION_ID Session Id of the user connecting.
| --option KEY=VALUE Key-value pair that is used to further configure the session.
""".stripMargin
// scalastyle:on line.size.limit

/**
* Parse the command line and configure the builder.
Expand Down
2 changes: 1 addition & 1 deletion docs/spark-connect-overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ The customizations may also be passed in through CLI arguments as shown below:
spark-connect-repl --host myhost.com --port 443 --token ABCDEFG
{% endhighlight %}

The supported list of CLI arguments may be found [here](https://github.com/apache/spark/blob/master/connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClientParser.scala#L48).

@panbingkun panbingkun Mar 13, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link has outdated.

The supported list of CLI arguments may be found [here](https://github.com/apache/spark/blob/master/connector/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClientParser.scala#L48).

#### Configure programmatically with a connection string

Expand Down