The SQL driver for Big Data JSON has many attributes that can be finetuned to improve handling in scenarios with unreliable network connections to the Big Data JSON server or high volumes of data. Also, many drivers have driver-specific attributes to finetune actual behaviour or handle data not matching specifications.
The Big Data JSON driver attributes are assigned a default value which seldom requires change. However, changes can be applied when needed on four levels, which are reflected in the table below by separate checkmarks:
- Connection string: the connection string from the settings*.xml file and applied during log on.
- Set SQL statement: a set SQL-statement to be executed once connection has been established.
- Log on: value to be specified interactively by user during log on in a user interface.
The connection string for Big Data JSON can be found in the settings*.xml file used for the database. The reference manuals contain instructions how to relocate the settings*.xml files. Settings*.xml files are typically located in the `%USERPROFILE%` folder in most deployment scenarios. Each data container of a database in the connection string can have a `connectionString` element specifying the name and values of attributes. Both name and value must be properly escaped according to XML-semantics. Actual application of the value is solely done during log on. A new connection must be established to change the value of a driver attribute using a connection string.
The set SQL statement can be executed after log on. The syntax is: `set NAME VALUE`, or for a distributed database: `set NAME@ALIAS VALUE`. In some scenarios you may need to enclose the driver attribute name in square brackets to escape it from parsing, for instance when a reserved SQL keyword is part of the name. The new value takes effect straight after execution of the set-statement. The set-statement can be executed as often as needed during a session.
Driver attributes that can be interactively set to a value are typically presented in the log on window. Depending on the platform and design decisions of the user interface designer, some or all of the available driver attributes can have been made available.
The Big Data JSON driver can be configured using the following attributes:
| Code | Description | Origin | Default Value | Set from Connection String | Set from Set SQL-Statement | Set from Drivers File | Set from Log On |
|---|---|---|:---:|:---:|:---:|:---:|:---:|
| `analysis-enforce-row-uniqueness` | Enforce rows to be unique for software analysis. A fingerprint is calculated from the whole row of data when the primary key column is unknown. | Shared | False | ✓ | ✓ | ✓ | |
| `bulk-delete-page-size-rows` | Number of rows to delete per batch when bulk deleting. | Shared | 10000 | ✓ | ✓ | ✓ | |
| `bulk-insert-page-size-bytes` | Approximate maximum size in bytes of batch when bulk inserting. | Shared | 10000000 | ✓ | ✓ | ✓ | |
| `bulk-insert-page-size-rows` | Number of rows to insert per batch when bulk inserting. | Shared | 10000 | ✓ | ✓ | ✓ | |
| `directories` | Directories | | c:\temp | ✓ | ✓ | ✓ | ✓ |
| `duplicate-row-mode` | Configure behaviour when multiple rows are discovered during a limited search based upon a fingerprint generated from the primary key value, or all column values when no primary key is known.
The following values can be used: "error" to raise error on duplicate fingerprint values in an API-page of rows, "keep" to ignore duplicate fingerprint values silently, "skip" like "keep" but with logging to trace of duplicate values or "null" for default behaviour (typically "error").. | | | ✓ | ✓ | ✓ | ✓ |
| `extension` | File Extension | | *.*json,*.*json.gz,*.*json.zst | ✓ | ✓ | ✓ | ✓ |
| `file-index-budget-percentage` | {res:itgen_pae_file_index_budget_percentage} | | 10 | ✓ | ✓ | ✓ | ✓ |
| `file-index-cache-directory` | {res:itgen_pae_file_index_cache_directory} | | C:\Users\guido\AppData\Local\Invantive\FileIndex | ✓ | ✓ | ✓ | ✓ |
| `file-index-directory` | {res:itgen_pae_file_index_directory} | | | ✓ | ✓ | ✓ | ✓ |
| `file-index-lock-expiry-min` | {res:itgen_pae_file_index_lock_expiry_min} | | 10 | ✓ | ✓ | ✓ | ✓ |
| `file-index-maintain` | {res:itgen_pae_file_index_maintain} | | True | ✓ | ✓ | ✓ | ✓ |
| `file-index-use` | {res:itgen_pae_file_index_use} | | True | ✓ | ✓ | ✓ | ✓ |
| `filter-while-reading` | {res:itgen_pae_filter_while_reading} | | True | ✓ | ✓ | ✓ | ✓ |
| `force-case-sensitive-identifiers` | Consider identifiers as case-sensitive independent of the platform capabilities. | Shared | False | ✓ | ✓ | ✓ | |
| `forced-casing-identifiers` | Forced casing of identifiers. Choose from: Unset, Lower, Upper and Mixed. | Shared | | ✓ | ✓ | ✓ | |
| `group-by-pushdown-max-groups` | {res:itgen_pae_group_by_pushdown_max_groups_desc} | Shared | 1000 | ✓ | ✓ | ✓ | |
| `ignore-erroneous-records` | {res:itgen_pae_ignore_erroneous_records} | | True | ✓ | ✓ | ✓ | ✓ |
| `ignore-unknown-path-type` | Whether to ignore path types not yet supported. An error will be generated when an unsupported type occurs. | | True | ✓ | ✓ | ✓ | ✓ |
| `ignore-values-unknown-path` | Whether to ignore values outside of processed paths. An error will be generated when a value occurs outside a path otherwise. | | True | ✓ | ✓ | ✓ | ✓ |
| `invantive-sql-compress-sparse-arrays` | Whether to compress sparse arrays in result sets during compression. | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-correct-invalid-date` | Whether to correct dates considered invalid since they are before 01-01-1753. When nullable, they are removed. Otherwise they are replaced by 01-01-1753. | SQL Engine V1 | False | ✓ | ✓ | ✓ | |
| `invantive-sql-execution-profile-disk-file-name` | Name of the file to write the execution profile of statements to. | SQL Engine V1 | profile-{date}-{applicationname}-{currentprocessid}-{sessionid}.ndjson | ✓ | ✓ | ✓ | |
| `invantive-sql-execution-profile-disk-path` | Folder to write the execution profile of statements to. | SQL Engine V1 | | ✓ | ✓ | ✓ | |
| `invantive-sql-execution-profile-to-disk` | Whether to write the execution profile of statements to disk. | SQL Engine V1 | False | ✓ | ✓ | ✓ | |
| `invantive-sql-forward-child-exists-filters-to-data-containers` | {res:itgen_pae_invantive_sql_forward_child_exists_filters_to_data_containers_desc} | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-forward-filters-to-data-containers` | Whether to forward filters to data containers. | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-forward-projections-to-data-containers` | {res:itgen_pae_invantive_sql_forward_projections_to_data_containers_desc} | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-forward-projections-to-synchronize-target` | {res:itgen_pae_invantive_sql_forward_projections_to_synchronize_target_desc} | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-psql-max-parallel` | {res:itgen_pae_invantive_sql_psql_max_parallel_desc} | SQL Engine V1 | 48 | ✓ | ✓ | ✓ | |
| `invantive-sql-share-byte-arrays` | Whether to share the memory used by identical byte arrays in result sets during compression. | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-share-strings` | Whether to share the memory used by identical strings in result sets during compression. | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `invantive-sql-shuffle-fetch-results-data-containers` | Whether to shuffle results fetched from data containers. | SQL Engine V1 | False | ✓ | ✓ | ✓ | |
| `invantive-use-cache` | Whether to cache the results of a query. | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `log-directory` | {res:itgen_pae_log_directory_desc} | | c:\temp | ✓ | ✓ | ✓ | |
| `log-native-calls-to-disk-max-events` | Maximum number of call events to register from last activation. | Shared | | ✓ | ✓ | ✓ | |
| `log-native-calls-to-disk-max-sec` | Maximum number of seconds to register calls from last activation. | Shared | | ✓ | ✓ | ✓ | |
| `log-native-calls-to-disk-on-error` | Registers native calls to data container backend as disk files when the call raised an error. | Shared | False | ✓ | ✓ | ✓ | |
| `log-native-calls-to-disk-on-success` | Registers native calls to data container backend as disk files when the call raised no error. | Shared | False | ✓ | ✓ | ✓ | |
| `log-native-calls-to-trace` | Log native calls to data container backend on the trace. | Shared | False | ✓ | ✓ | ✓ | |
| `log-text` | {res:itgen_pae_log_text_desc} | | False | ✓ | ✓ | ✓ | |
| `max-discovered-column-count` | Maximum number of discovered columns. An error will be generated when the column exceeds this value. | | 250 | ✓ | ✓ | ✓ | ✓ |
| `max-erroneous-files` | Maximum allowed number of files with errors before an error is raised. | | 0 | ✓ | ✓ | ✓ | ✓ |
| `max-length-identifiers` | Non-default maximum length in characters of identifier names. | Shared | | ✓ | ✓ | ✓ | |
| `partition-slot-based-rate-limit-length-ms` | Total length in milliseconds across all slots of a partition-based rate limit. | Shared | 60000 | ✓ | | ✓ | |
| `partition-slot-based-rate-limit-slots` | Number of slots per partition-based rate limit. Null means no slot-based rate limit. | Shared | | ✓ | | ✓ | |
| `pre-request-delay-ms` | Pre-request delay in milliseconds per request. | Shared | 0 | ✓ | ✓ | ✓ | |
| `preemptive-loading-blobs-per-minute` | {res:itgen_pae_preemptive_loading_blobs_per_minute_desc} | SQL Engine V1 | 20 | ✓ | ✓ | ✓ | |
| `preemptive-loading-daily-reserve-fraction` | {res:itgen_pae_preemptive_loading_daily_reserve_fraction_desc} | SQL Engine V1 | 0,4 | ✓ | ✓ | ✓ | |
| `preemptive-loading-disk-budget-mb` | {res:itgen_pae_preemptive_loading_disk_budget_mb_desc} | SQL Engine V1 | 8192 | ✓ | ✓ | ✓ | |
| `preemptive-loading-first-use-rows` | {res:itgen_pae_preemptive_loading_first_use_rows_desc} | SQL Engine V1 | 1000 | ✓ | ✓ | ✓ | |
| `preemptive-loading-horizon-min` | {res:itgen_pae_preemptive_loading_horizon_min_desc} | SQL Engine V1 | 60 | ✓ | ✓ | ✓ | |
| `preemptive-loading-idle-scope-data-container` | {res:itgen_pae_preemptive_loading_idle_scope_data_container_desc} | SQL Engine V1 | False | ✓ | ✓ | ✓ | |
| `preemptive-loading-idle-sec` | {res:itgen_pae_preemptive_loading_idle_sec_desc} | SQL Engine V1 | 5 | ✓ | ✓ | ✓ | |
| `preemptive-loading-journal` | {res:itgen_pae_preemptive_loading_journal_desc} | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `preemptive-loading-max-parallel` | {res:itgen_pae_preemptive_loading_max_parallel_desc} | SQL Engine V1 | 1 | ✓ | ✓ | ✓ | |
| `preemptive-loading-max-unattended-min` | {res:itgen_pae_preemptive_loading_max_unattended_min_desc} | SQL Engine V1 | 120 | ✓ | ✓ | ✓ | |
| `preemptive-loading-node-max-parallel` | {res:itgen_pae_preemptive_loading_node_max_parallel_desc} | SQL Engine V1 | 4 | ✓ | ✓ | ✓ | |
| `preemptive-loading-rate-fraction` | {res:itgen_pae_preemptive_loading_rate_fraction_desc} | SQL Engine V1 | 0,75 | ✓ | ✓ | ✓ | |
| `preemptive-loading-serve-stale-max-age-sec` | {res:itgen_pae_preemptive_loading_serve_stale_max_age_sec_desc} | SQL Engine V1 | | ✓ | ✓ | ✓ | |
| `preemptive-loading-superset-max-rows` | {res:itgen_pae_preemptive_loading_superset_max_rows_desc} | SQL Engine V1 | 250000 | ✓ | ✓ | ✓ | |
| `preemptive-loading-superset` | {res:itgen_pae_preemptive_loading_superset_desc} | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `preemptive-loading` | {res:itgen_pae_preemptive_loading_desc} | SQL Engine V1 | True | ✓ | ✓ | ✓ | |
| `read-columns-in-use-only` | {res:itgen_pae_read_columns_in_use_only} | | True | ✓ | ✓ | ✓ | ✓ |
| `recurse-directories` | {res:itgen_pae_recurse_directories} | | True | ✓ | ✓ | ✓ | ✓ |
| `report-unmapped-values` | {res:itgen_pae_report_unmapped_values} | | False | ✓ | ✓ | ✓ | ✓ |
| `requested-maximum-usable-partitions` | Requested Maximum Usable Partitions | Shared | | ✓ | ✓ | ✓ | |
| `requested-page-size` | Preferred number of rows to exchange per round trip; only effective on limited platforms such as AFAS Online. | Shared | | ✓ | ✓ | ✓ | |
| `requests-parallel-max` | Maximum number of parallel data requests from individual partitions on the data container. | Shared | 32 | ✓ | ✓ | ✓ | |
| `slot-based-rate-limit-length-ms` | Total length in milliseconds across all slots of a slot-based rate limit. | Shared | 60000 | ✓ | | ✓ | |
| `slot-based-rate-limit-slots` | Number of slots of a slot-based rate limit. Null means no slot-based rate limit. | Shared | | ✓ | | ✓ | |
| `standardize-identifiers-casing` | Rewrite all identifiers to the recommended standard platform-specific casing when changing a data model on a case-dependent platform. | Shared | True | ✓ | ✓ | ✓ | |
| `standardize-identifiers` | Rewrite all identifiers to the preferred standards as configured by standardize-identifiers-casing and maximum-length-identifiers. | Shared | True | ✓ | ✓ | ✓ | |
| `swagger-specification-files` | Comma-separated list of Swagger specification files. | | | ✓ | ✓ | ✓ | ✓ |