SQL `EXPORT DATABASE` only works from console not via HTTP API

[gramian]

ArcadeDB Version: v22.10.1 (build e692f4c/1667424782661/main)

JDK Version: openjdk 11.0.16.1 2022-08-12

OS: MacOS 12.6

Expected behavior

Database exported successfully

result: ok

Actual behavior

Database 'tmp' is not available

Trace

<ArcadeDB_0> Error on command execution (PostCommandHandler)
com.arcadedb.exception.DatabaseIsClosedException: Database 'tmp' is not available
	at com.arcadedb.server.ArcadeDBServer.getDatabase(ArcadeDBServer.java:357)
	at com.arcadedb.server.http.handler.DatabaseAbstractHandler.execute(DatabaseAbstractHandler.java:59)
	at com.arcadedb.server.http.handler.AbstractHandler.handleRequest(AbstractHandler.java:111)
	at io.undertow.server.RoutingHandler.handleRequest(RoutingHandler.java:93)
	at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:104)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:391)
	at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:256)
	at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
	at io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:162)
	at io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:100)
	at io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:57)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
	at org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:291)
	at org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:286)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
	at org.xnio.nio.QueuedNioTcpServer2.acceptTask(QueuedNioTcpServer2.java:178)
	at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:479)

Steps to reproduce

EXPORT DATABASE file://database.jsonl.tgz

via Studio or curl.

Furthermore

I also propose that the url argument becomes optional, and by default a similar filename as in BACKUP DATABASE is used, ie: dbname-export-YYYYMMDD-HHmmssuuu.zip.

[gramian]

@lvca This seems to work meanwhile when used without the FORMAT 'JSONL' statement. Using FORMAT yields:

Cannot execute command Encountered " <STRING_LITERAL> " 'JSONL ' "" at line 1, column 50. Was expecting one of: <ALIGN> ... <CUSTOM> ... <TO> ... <VALUE> ... <VALUES> ... <SET> ... <ADD> ... <PUT> ... <MERGE> ... <CONTENT> ... <REMOVE> ... <OFFSET> ... <RECORD> ... <CHECK> ... <NEAR> ... <WITH> ... <WITHIN> ... <MINDEPTH> ... <TYPE> ... <SUPERTYPE> ... <TYPES> ... <SUPERTYPES> ... <EXCEPTION> ... <PROFILE> ... <ON> ... <OFF> ... <TRUNCATE> ... <FIND> ... <EXTENDS> ... <BUCKETS> ... <BUCKETSELECTIONSTRATEGY> ... <ALTER> ... <NAME> ... <ADDBUCKET> ... <REMOVEBUCKET> ... <DROP> ... <PROPERTY> ... <FORCE> ... <SCHEMA> ... <INDEX> ... <ENGINE> ... <REBUILD> ... <FORMAT> ... <EXPORT> ... <IMPORT> ... <DATABASE> ... <OPTIMIZE> ... <LINK> ... <INVERSE> ... <EXPLAIN> ... <GRANT> ... <REVOKE> ... <READ> ... <EXECUTE> ... <ALL> ... <NONE> ... <FUNCTION> ... <PARAMETERS> ... <LANGUAGE> ... <BEGIN> ... <COMMIT> ... <ROLLBACK> ... <IF> ... <ELSE> ... <CONTINUE> ... <FAIL> ... <ISOLATION> ... <SLEEP> ... <CONSOLE> ... <START> ... <OPTIONAL> ... <COUNT> ... <EXISTS> ... <MOVE> ... <DEPTH_ALIAS> ... <PATH_ALIAS> ... <IDENTIFIED> ... <RID> ... <SYSTEM> ... <IN> ... <KEY> ... <IDENTIFIER> ... <QUOTED_IDENTIFIER> ... 

Details

<ArcadeDB_0> Error on command execution (PostCommandHandler)
com.arcadedb.exception.CommandSQLParsingException: export database file://database.json.tgz FORMAT 'JSONL'
	at com.arcadedb.query.sql.parser.StatementCache.throwParsingException(StatementCache.java:126)
	at com.arcadedb.query.sql.parser.StatementCache.parse(StatementCache.java:116)
	at com.arcadedb.query.sql.parser.StatementCache.get(StatementCache.java:70)
	at com.arcadedb.query.sql.SQLQueryEngine.parse(SQLQueryEngine.java:196)
	at com.arcadedb.query.sql.SQLQueryEngine.command(SQLQueryEngine.java:95)
	at com.arcadedb.database.EmbeddedDatabase.command(EmbeddedDatabase.java:1236)
	at com.arcadedb.server.ServerDatabase.command(ServerDatabase.java:416)
	at com.arcadedb.server.http.handler.PostCommandHandler.executeCommand(PostCommandHandler.java:127)
	at com.arcadedb.server.http.handler.PostCommandHandler.execute(PostCommandHandler.java:88)
	at com.arcadedb.server.http.handler.DatabaseAbstractHandler.execute(DatabaseAbstractHandler.java:92)
	at com.arcadedb.server.http.handler.AbstractHandler.handleRequest(AbstractHandler.java:128)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.arcadedb.query.sql.parser.ParseException: Encountered " <STRING_LITERAL> "\'JSONL\' "" at line 1, column 50.
Was expecting one of:
    <ALIGN> ...
    <CUSTOM> ...
    <TO> ...
    <VALUE> ...
    <VALUES> ...
    <SET> ...
    <ADD> ...
    <PUT> ...
    <MERGE> ...
    <CONTENT> ...
    <REMOVE> ...
    <OFFSET> ...
    <RECORD> ...
    <CHECK> ...
    <NEAR> ...
    <WITH> ...
    <WITHIN> ...
    <MINDEPTH> ...
    <TYPE> ...
    <SUPERTYPE> ...
    <TYPES> ...
    <SUPERTYPES> ...
    <EXCEPTION> ...
    <PROFILE> ...
    <ON> ...
    <OFF> ...
    <TRUNCATE> ...
    <FIND> ...
    <EXTENDS> ...
    <BUCKETS> ...
    <BUCKETSELECTIONSTRATEGY> ...
    <ALTER> ...
    <NAME> ...
    <ADDBUCKET> ...
    <REMOVEBUCKET> ...
    <DROP> ...
    <PROPERTY> ...
    <FORCE> ...
    <SCHEMA> ...
    <INDEX> ...
    <ENGINE> ...
    <REBUILD> ...
    <FORMAT> ...
    <EXPORT> ...
    <IMPORT> ...
    <DATABASE> ...
    <OPTIMIZE> ...
    <LINK> ...
    <INVERSE> ...
    <EXPLAIN> ...
    <GRANT> ...
    <REVOKE> ...
    <READ> ...
    <EXECUTE> ...
    <ALL> ...
    <NONE> ...
    <FUNCTION> ...
    <PARAMETERS> ...
    <LANGUAGE> ...
    <BEGIN> ...
    <COMMIT> ...
    <ROLLBACK> ...
    <IF> ...
    <ELSE> ...
    <CONTINUE> ...
    <FAIL> ...
    <ISOLATION> ...
    <SLEEP> ...
    <CONSOLE> ...
    <START> ...
    <OPTIONAL> ...
    <COUNT> ...
    <EXISTS> ...
    <MOVE> ...
    <DEPTH_ALIAS> ...
    <PATH_ALIAS> ...
    <IDENTIFIED> ...
    <RID> ...
    <SYSTEM> ...
    <IN> ...
    <KEY> ...
    <IDENTIFIER> ...
    <QUOTED_IDENTIFIER> ...
    
	at com.arcadedb.query.sql.parser.SqlParser.generateParseException(SqlParser.java:26995)
	at com.arcadedb.query.sql.parser.SqlParser.jj_consume_token(SqlParser.java:26840)
	at com.arcadedb.query.sql.parser.SqlParser.Identifier(SqlParser.java:911)
	at com.arcadedb.query.sql.parser.SqlParser.ExportDatabaseStatement(SqlParser.java:16879)
	at com.arcadedb.query.sql.parser.SqlParser.StatementInternal(SqlParser.java:1383)
	at com.arcadedb.query.sql.parser.SqlParser.Statement(SqlParser.java:1044)
	at com.arcadedb.query.sql.parser.SqlParser.Parse(SqlParser.java:245)
	at com.arcadedb.query.sql.parser.StatementCache.parse(StatementCache.java:111)
	... 16 more

The same holds for GRAPHML or GRAPHSON.

What do you think about my proposal of making the url parameter optional (see above)?

PS: Note the space at the end of JSONL in the error message (I input JSONL without the space): Cannot execute command Encountered " <STRING_LITERAL> " 'JSONL ' "" at line 1, column 50

[lvca]

@gramian is this issue still relevant?

[gramian]

@lvca yes, here is the current state:

• FORMAT JSONL works (note absence of quotation marks around JSONL), I will update the docu and remove the quatation marks
• FORMAT GRAPHML throws an error even without quotation marks: Error on command execution (PostCommandHandler)

Trace

``` com.arcadedb.exception.CommandExecutionException: Error on exporting database at com.arcadedb.query.sql.parser.ExportDatabaseStatement.executeSimple(ExportDatabaseStatement.java:86) at com.arcadedb.query.sql.executor.SingleOpExecutionPlan.executeInternal(SingleOpExecutionPlan.java:91) at com.arcadedb.query.sql.parser.SimpleExecStatement.execute(SimpleExecStatement.java:63) at com.arcadedb.query.sql.parser.Statement.execute(Statement.java:85) at com.arcadedb.query.sql.parser.Statement.execute(Statement.java:69) at com.arcadedb.query.sql.SQLQueryEngine.command(SQLQueryEngine.java:97) at com.arcadedb.database.EmbeddedDatabase.command(EmbeddedDatabase.java:1264) at com.arcadedb.server.ServerDatabase.command(ServerDatabase.java:421) at com.arcadedb.server.http.handler.PostCommandHandler.executeCommand(PostCommandHandler.java:127) at com.arcadedb.server.http.handler.PostCommandHandler.execute(PostCommandHandler.java:88) at com.arcadedb.server.http.handler.DatabaseAbstractHandler.execute(DatabaseAbstractHandler.java:92) at com.arcadedb.server.http.handler.AbstractHandler.handleRequest(AbstractHandler.java:127) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859) at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: com.arcadedb.integration.exporter.ExportException: Error on writing to 'exports/database.graphml.tgz' at com.arcadedb.integration.exporter.Exporter.exportDatabase(Exporter.java:103) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.arcadedb.query.sql.parser.ExportDatabaseStatement.executeSimple(ExportDatabaseStatement.java:79) ... 18 more Caused by: com.arcadedb.integration.exporter.ExportException: Transaction in progress found at com.arcadedb.gremlin.integration.exporter.format.GraphMLExporterFormat.exportDatabase(GraphMLExporterFormat.java:56) at com.arcadedb.integration.exporter.Exporter.exportDatabase(Exporter.java:80) ... 23 more ```

• FORMAT GRAPHSON throws an error even without quotation marks: Error on command execution (PostCommandHandler)

Trace

``` com.arcadedb.exception.CommandExecutionException: Error on exporting database at com.arcadedb.query.sql.parser.ExportDatabaseStatement.executeSimple(ExportDatabaseStatement.java:86) at com.arcadedb.query.sql.executor.SingleOpExecutionPlan.executeInternal(SingleOpExecutionPlan.java:91) at com.arcadedb.query.sql.parser.SimpleExecStatement.execute(SimpleExecStatement.java:63) at com.arcadedb.query.sql.parser.Statement.execute(Statement.java:85) at com.arcadedb.query.sql.parser.Statement.execute(Statement.java:69) at com.arcadedb.query.sql.SQLQueryEngine.command(SQLQueryEngine.java:97) at com.arcadedb.database.EmbeddedDatabase.command(EmbeddedDatabase.java:1264) at com.arcadedb.server.ServerDatabase.command(ServerDatabase.java:421) at com.arcadedb.server.http.handler.PostCommandHandler.executeCommand(PostCommandHandler.java:127) at com.arcadedb.server.http.handler.PostCommandHandler.execute(PostCommandHandler.java:88) at com.arcadedb.server.http.handler.DatabaseAbstractHandler.execute(DatabaseAbstractHandler.java:92) at com.arcadedb.server.http.handler.AbstractHandler.handleRequest(AbstractHandler.java:127) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859) at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: com.arcadedb.integration.exporter.ExportException: Error on writing to 'exports/database.graphson.tgz' at com.arcadedb.integration.exporter.Exporter.exportDatabase(Exporter.java:103) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.arcadedb.query.sql.parser.ExportDatabaseStatement.executeSimple(ExportDatabaseStatement.java:79) ... 18 more Caused by: com.arcadedb.integration.exporter.ExportException: Transaction in progress found at com.arcadedb.gremlin.integration.exporter.format.GraphSONExporterFormat.exportDatabase(GraphSONExporterFormat.java:56) at com.arcadedb.integration.exporter.Exporter.exportDatabase(Exporter.java:80) ... 23 more ```

• default name when no url or filename is given to dbname-export-YYYYMMDD-HHmmssuuu.zip

[lvca]

Fixed the issue, now JSONL, GraphSON and GraphML work.