feat(snowflake-connectors): meetings implementation [CM-1034]#3998
feat(snowflake-connectors): meetings implementation [CM-1034]#3998
Conversation
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
|
|
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
There was a problem hiding this comment.
Pull request overview
Adds “meetings” platform support to the Snowflake connectors pipeline, including new activity types for meeting invitations and attendance, plus the necessary enum/config updates so meetings is recognized across the codebase.
Changes:
- Introduces
MEETINGSas a supported platform/org source and wires it into source priority. - Adds meetings activity types (
invited-meeting,attended-meeting) with scoring grid + DB registration. - Extends the transformer pipeline to allow a single row to produce multiple activities and updates the consumer to emit them all.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| services/libs/types/src/enums/platforms.ts | Adds MEETINGS to PlatformType. |
| services/libs/types/src/enums/organizations.ts | Adds MEETINGS to organization source enums. |
| services/libs/integrations/src/integrations/meetings/types.ts | Defines meetings activity types + scoring grid. |
| services/libs/integrations/src/integrations/index.ts | Exports meetings integration types. |
| services/libs/data-access-layer/src/organizations/attributesConfig.ts | Adds meetings to org attribute source priority list. |
| services/apps/snowflake_connectors/src/integrations/types.ts | Registers meetings meeting-attendance as a data source name. |
| services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts | Implements meetings attendance transformer producing invited/attended activities. |
| services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/buildSourceQuery.ts | Adds Snowflake query for exporting meeting attendance rows + incremental logic. |
| services/apps/snowflake_connectors/src/integrations/index.ts | Registers meetings platform and its source in the connectors registry. |
| services/apps/snowflake_connectors/src/core/transformerBase.ts | Updates transformer API to allow multiple activities per row; normalizes safeTransformRow to arrays. |
| services/apps/snowflake_connectors/src/consumer/transformerConsumer.ts | Updates consumer to handle multiple transformed activities per input row. |
| backend/src/database/migrations/V1775219382__addMeetingsActivityTypes.sql | Inserts new meetings activity types into activityTypes. |
| backend/src/database/migrations/U1775219382__addMeetingsActivityTypes.sql | (Present but empty) rollback migration file for the above insert. |
| .claude/skills/scaffold-snowflake-connector/SKILL.md | Updates internal scaffolding guidance related to org mapping discovery. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
...s/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/buildSourceQuery.ts
Outdated
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Show resolved
Hide resolved
| for (const result of results) { | ||
| const resolved = await this.integrationResolver.resolve(platform, result.segment) | ||
| if (!resolved) { | ||
| resolveSkippedCount++ | ||
| continue | ||
| } | ||
|
|
||
| await this.emitter.createAndProcessActivityResult( | ||
| resolved.segmentId, | ||
| resolved.integrationId, | ||
| result.activity, | ||
| ) | ||
| transformedCount++ | ||
| } |
There was a problem hiding this comment.
Now that transformers may return multiple activities per row, this loop resolves the same segment repeatedly when multiple activities share it (common when a row yields multiple activity types). Consider de-duplicating integrationResolver.resolve(...) calls per row (e.g., cache by slug+sourceId) to avoid extra Redis/DB lookups and reduce latency.
backend/src/database/migrations/V1775219382__addMeetingsActivityTypes.sql
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Show resolved
Hide resolved
| platform: PlatformType.MEETINGS, | ||
| timestamp, | ||
| score: MEETINGS_GRID[MeetingsActivityType.INVITED_MEETING].score, | ||
| sourceId: `${primaryKey}_invited`, |
There was a problem hiding this comment.
@joanagmaia, this one is different from specs, as meeting_id isn't unique (similar to committees). Used primaryKey as it seems to be unique
...s/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/buildSourceQuery.ts
Outdated
Show resolved
Hide resolved
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
| : null | ||
| if (!date || isNaN(date.getTime())) return null | ||
| if (typeof rawTime === 'number') { | ||
| date.setTime(date.getTime() + rawTime) |
There was a problem hiding this comment.
Parquet TIME value unit assumption may produce wrong timestamps
High Severity
toISOTimestamp adds rawTime directly to the date via date.setTime(date.getTime() + rawTime), assuming the Parquet TIME column value is in milliseconds. Snowflake typically exports TIME columns to Parquet using the TIME_MICROS logical type (microseconds since midnight). If @dsnp/parquetjs returns the raw microsecond value as a number, every meeting timestamp would be off by a factor of 1000 — e.g., a 10 AM meeting would appear ~416 days in the future. This is the first transformer in the codebase to handle a separate TIME column, so there's no prior precedent to validate the assumption.
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Show resolved
Hide resolved
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
...s/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/buildSourceQuery.ts
Outdated
Show resolved
Hide resolved
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Outdated
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Outdated
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/meetings/meeting-attendance/transformer.ts
Show resolved
Hide resolved
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
|
|
||
| if (!accountName && !website && !domainAliases) { | ||
| return undefined | ||
| } |
There was a problem hiding this comment.
Organizations created without identities when only account name exists
Medium Severity
The buildOrganizations guard (!accountName && !website && !domainAliases) differs from the established pattern in Cvent/TNC, which only create orgs when domain data exists (!website && !domainAliases). When only ACCOUNT_NAME from the main meeting table is present but the org LEFT JOIN produces no match, an organization is created with a displayName but an empty identities array, no domain, no logo, and no industry. These identity-less org records cannot be merged with other org records for the same company, leading to duplicate organization entries.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 188c6c5. Configure here.
There was a problem hiding this comment.
@mbani01 also valid feedback if we are attempting to create organizations without identities -> this will fail
There was a problem hiding this comment.
Forgot this one, will get back to it tomorrow
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mbani@contractor.linuxfoundation.org>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e7b8669. Configure here.
| { | ||
| name: DataSourceName.MEETINGS_MEETING_ATTENDANCE, | ||
| buildSourceQuery: meetingAttendanceBuildQuery, | ||
| transformer: new MeetingAttendanceTransformer(), |
There was a problem hiding this comment.
MEETINGS entry missing closing brackets in platform registry
High Severity
The [PlatformType.MEETINGS] entry in the supported object is missing its closing }, ], and }, after the transformer line. Line 32 (transformer: new MeetingAttendanceTransformer(),) is immediately followed by [PlatformType.COMMITTEES] on line 33 without any closing brackets. This causes all subsequent platform definitions (COMMITTEES, CVENT, TNC) to be parsed as nested computed properties inside the MEETINGS source object rather than as top-level entries in supported. This breaks the entire platform registry — no platform will resolve correctly at runtime.
Reviewed by Cursor Bugbot for commit e7b8669. Configure here.


This pull request adds support for processing and tracking meeting attendance activities from the "meetings" platform. It introduces new activity types for meeting invitations and attendance, integrates the meetings data source into the Snowflake connectors pipeline, and updates relevant enums and configurations to recognize meetings as a valid platform and organization source.
Meetings platform support:
MEETINGSas a newPlatformType,OrganizationSource, andOrganizationAttributeSource, and updated the source priority list to include meetings. [1] [2] [3] [4]invited-meeting,attended-meeting) and their scoring grid, and registered them in the database. [1] [2]Snowflake connectors integration:
MeetingAttendanceTransformerand its source query, and registered the source in the Snowflake connectors integration index. [1] [2] [3] [4] [5]Transformer pipeline improvements:
transformRowandsafeTransformRownow handle arrays of activities, and the consumer processes all returned activities. [1] [2] [3] [4]Integrations exports:
Note
Medium Risk
Adds a new Snowflake connector source plus new enums/migrations, and changes the transformation pipeline to process multiple activities per input row, which can affect downstream activity volume/dedup behavior if any transformer starts returning arrays unexpectedly.
Overview
Adds a new
meetingsSnowflake connector source for meeting attendance, includingbuildSourceQuery(segment matching, org lookup, incremental + dedup logic) and aMeetingAttendanceTransformerthat can emit invited and/or attended meeting activities from a single exported row.Extends the Snowflake connectors framework to support multiple activities per row by updating
TransformerBase.transformRow/safeTransformRowto allow arrays and updatingTransformerConsumerto iterate and emit each result.Registers the new platform and activity types across the stack:
PlatformType.MEETINGS, meetings activity types + scoring grid (exported from@crowd/integrations), Flyway migration to insert newactivityTypes, and organization source/attribute source updates (including org attribute source priority) to recognizeMEETINGS.Reviewed by Cursor Bugbot for commit e7b8669. Bugbot is set up for automated code reviews on this repo. Configure here.