Skip to content

Add support for more types in gql - #464

Merged
gledis69 merged 14 commits into
mainfrom
graphql-add-more-types
Jun 16, 2022
Merged

Add support for more types in gql#464
gledis69 merged 14 commits into
mainfrom
graphql-add-more-types

Conversation

@gledis69

@gledis69 gledis69 commented May 30, 2022

Copy link
Copy Markdown
Contributor

Done:

  • Byte (not supported by Pg)
  • Short
  • Long
  • Single
  • Decimal
  • DateTime
    This is a bit of a tricky one. DateTime type is handled by HotChocolate as DateTimeOffset. Also, our db column type detection method gives DateTime for all Date, DateTime, and DateTimeOffset. Meaning that while the user can define non DateTimeOffset columns, they will be interpreted as DateTimeOffset and the gaps will be filled with 00:00:00 time and local offset.
    HotChocolate also provides us with Date type. However, because all db column types Date, DateTime, and DateTimeOffset are detected as DateTime system types, we cannot make use of the HotChocolate Date type as of now.
  • ByteArray
    A base64 representation of the byte array is expected to be input to the queries and similarly the base64 representation of the byte array will be returned.
    MsSql does not support inserting/updating a bytearray (varbinary) to null due to: https://stackoverflow.com/questions/29254690/why-does-dbnull-value-require-a-proper-sqldbtype
    Essentially, when passing parameters to the db, the parameters are automatically cast to string. But varbinary does not accept a string even though all other column types have not complained about this across all three sql db-s 😅 . This will likely require us to change the way we construct the parameter dictionary to include the db column type. This change is relatively large and out of the scope of this PR.
    Opened an issue for this: Sql: Specify parameter type when passing parameters to the sql conn #475

Todos (might consider doing in a separate PR-s):

  • Supported Type docs
  • ODataFilter tests for the new types
  • GQL filter and orderby tests for the new types

The changes required to add a new type supported by HotChocolate:

  • At: GraphQLTypes.DefaultValueType.Configure: specify the type of the value the default value directive takes for each type.
  • At GraphQLUtils.IsBuiltInType: add the type to built in types so it is not parsed as if it is a model type (aka matched a table in the database).
  • At StandardQueryInputs: create a filter input type for the new type and add that input type to StandardQueryInputs.InputTypes.
  • At EdmModelBuilder.BuildEntityTypes: add the type to the switch statement for type resolution.
  • At SchemaConverter.FromTableDefinition: create the appropriate ObjectValueNode when the column with the type introduced has a default value.
  • At SchemaConverter.GetGraphQLTypeForColumnType: add the new type to the switch statement.
  • At BaseSqlQueryStructure.GetParamAsColumnSystemType: add the type to the switch statement.

If the type is not support supported by HotChocolate (e.g. Single):

  • Create a custom type (e.g. SingleType defined in GraphQLBuilder.CustomTypes)
  • Do the steps above

@gledis69
gledis69 force-pushed the graphql-add-more-types branch from a760148 to 03c3f71 Compare May 30, 2022 14:49
@gledis69
gledis69 force-pushed the graphql-add-more-types branch from 3d463b1 to fb5387e Compare June 3, 2022 19:25
Comment thread DataGateway.Service.Tests/SqlTests/MsSqlGQLSupportedTypesTests.cs Outdated
jsonColumns.Add($"{parametrizedCLabel}, CAST({subqueryName}.{QuoteIdentifier(cLabel)} is true as json)");
}
else if (column.ColumnName != SqlQueryStructure.DATA_IDENT &&
structure.GetColumnSystemType(column.ColumnName) == typeof(byte[]))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we use is operator here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is requires a constant on the other end and typeof(byte[]) is not.

Comment thread DataGateway.Service/Resolvers/PostgresQueryBuilder.cs

@aaronpowell aaronpowell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit on one test, otherwsie looks good (and yay for wider type support!)

{
if (!IsTypeSupportedType(type, value))
{
return;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this fail the test? Otherwise we won't know if we're running the test for types that aren't supported.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failing would cause the CI to fail and the PR wouldn't be merged.
I ended up settling for Assert.Inconclusive. It skips the test when executed.

INSERT INTO brokers("ID Number", "First Name", "Last Name") VALUES (1, 'Michael', 'Burry'), (2, 'Jordan', 'Belfort');
INSERT INTO stocks_price(categoryid, pieceid, instant, price, is_wholesale_price) VALUES (2, 1, 'instant1', 100.57, True), (1, 1, 'instant2', 42.75, False);
INSERT INTO type_table(id, int_types, string_types, float_types, boolean_types) VALUES (1, 1, '', 0.33, true), (2, -1, 'lksa;jdflasdf;alsdflksdfkldj', -9.2, false), (3, 123456, 'null', 1555.99, true), (4, NULL, NULL, NULL, NULL);
INSERT INTO type_table(id, short_types, int_types, long_types, string_types, single_types, float_types, decimal_types, boolean_types, datetime_types, bytearray_types) VALUES

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make sure we test min/max value for each type?

Comment thread DataGateway.Service/Services/ResolverMiddleware.cs Outdated
short_types smallint,
int_types int,
long_types bigint,
string_types varchar(max),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wonder if we support nvarchar(max)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup I checked.
varchar and nvarchar are both detected as string types, and they are handled accordingly.
HotChocolate supports unicode as well it seems. So nvarchar is fully supported

Comment thread DataGateway.Service.Tests/SqlTests/PostgreSqlGQLSupportedTypesTests.cs Outdated
Comment thread DataGateway.Service.Tests/SqlTests/MsSqlGQLSupportedTypesTests.cs Outdated
return false;
"ID",
"Byte",
"Short",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we use something like SingleType.TypeName

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this instead of the "Single" string. Short corresponds to small int.

Comment thread DataGateway.Service.GraphQLBuilder/Queries/StandardQueryInputs.cs Outdated
new InputValueDefinitionNode(null, new NameNode("lt"), new StringValueNode("Less Than"), new ByteType().ToTypeNode(), null, new List<DirectiveNode>()),
new InputValueDefinitionNode(null, new NameNode("lte"), new StringValueNode("Less Than or Equal To"), new ByteType().ToTypeNode(), null, new List<DirectiveNode>()),
new InputValueDefinitionNode(null, new NameNode("neq"), new StringValueNode("Not Equals"), new ByteType().ToTypeNode(), null, new List<DirectiveNode>()),
new InputValueDefinitionNode(null, new NameNode("isNull"), new StringValueNode("Not null test"), new BooleanType().ToTypeNode(), null, new List<DirectiveNode>())

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isNull or isNotNull?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isNull was used for other types as well, and our filter parsing expects isNull.

protected abstract string MakeQueryOnTypeTable(List<string> columnsToQuery, int id);
protected virtual bool IsTypeSupportedType(string type, string value = null)
{
return true;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

throw not implemented? so that we dont use it accidentally?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up using Assert.Inconclusive which skips the test. Throwing not implemented will fail the test which would block the PR from merging.
This is not technically just not implemented.
e.g. Postgres does not support byte type. That will never be implemented. The case is a bit different for varbinary for MsSql which we need to support at some point

Comment thread DataGateway.Service.Tests/SqlTests/GraphQLSupportedTypesTestsBase.cs Outdated
switch (floatType)
{
case SINGLE_TYPE:
Assert.AreEqual(float.Parse(expectedFloat), float.Parse(actualFloat));

@Aniruddh25 Aniruddh25 Jun 6, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is float? An instance, a class and is it a member variable? Could you correct its case so it can be easily identified from the name?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiler complains about Float.parse. float is a variable type in this case, and float.Parse is consistent with our other conversions: int.Parse, long.Parse etc.

@Aniruddh25 Aniruddh25 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for widening our db types support. This is a very nice enhancement.

LGTM after resolving suggestions and some questions!

@gledis69
gledis69 merged commit 63d5ef2 into main Jun 16, 2022
@gledis69
gledis69 deleted the graphql-add-more-types branch June 16, 2022 17:09
@ayush3797
ayush3797 restored the graphql-add-more-types branch August 16, 2023 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants