Limits on the Number of Destination Columns
Some Destinations impose a hard limit on the number of columns that can be replicated to a table. For some other Destinations, Hevo may restrict the number of columns in order to optimize the data replication process. Further, Hevo reserves six columns for metadata information it may append to a table during replication. Therefore, the final number of columns that you can map and replicate is derived as:
Number of Columns = Destination Limit - 6 Hevo Metadata Columns
Destination Name | Number of Columns | Limit Imposed By |
---|---|---|
Amazon Aurora MySQL | 4096-6 = 4090 | Hevo |
Amazon Redshift | 1600-6 = 1594 | Destination |
Google BigQuery | 4096-6 = 4090 | Hevo |
MS SQL (all variants except Amazon Aurora MS SQL) | 1024-6 = 1018 | Destination |
MySQL (all variants except Amazon Aurora MySQL) | 4096-6 = 4090 | Destination |
PostgreSQL (all variants except Amazon Aurora PostgreSQL) | 1600-6 = 1594 | Destination |
Snowflake | 4096-6 = 4090 | Hevo |
If the Source Event Type (object) has X number of fields and the Destination table only allows Y, where Y < X number of columns, then the Events of excess columns are sidelined with the error:
“Source Event Type has more than X-6 fields. Please remove the fields that are not required via Transformations and reset the mappings for the Event Type from Schema Mapper.”
If you have selected the Merge Tables or Merge Collections option in the Source configuration to merge tables or collections with the same name, then:
-
The columns of the table that is picked first are mapped first. Subsequently, columns from other tables are mapped. If the aggregate of the columns of all tables being merged is within the Destination’s limits, no error is displayed.
-
If the tables have columns with same names, the columns of the table that is picked first are mapped first. In case the data types of columns of subsequent tables do not match, their Events are sidelined.
-
If all tables individually have the maximum allowed columns or more, then the same error as above is displayed.
-
If the first table’s columns exceed the limit, while the next table has lesser column, still the table read first is considered first.