Merging upstream version 25.16.1.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
7688e2bdf8
commit
bad79d1f7c
110 changed files with 75353 additions and 68092 deletions
253
CHANGELOG.md
253
CHANGELOG.md
|
@ -1,6 +1,247 @@
|
|||
Changelog
|
||||
=========
|
||||
|
||||
## [v25.16.0] - 2024-08-22
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`f68d155`](https://github.com/tobymao/sqlglot/commit/f68d155c38a79a6527685c37f8de8773ce790bca) - exp.Merge, for Trino and Postgres, dont strip the target alias from then WHEN MATCHED condition to prevent an ambiguous column error *(PR [#3940](https://github.com/tobymao/sqlglot/pull/3940) by [@erindru](https://github.com/erindru))*:
|
||||
|
||||
exp.Merge, for Trino and Postgres, dont strip the target alias from then WHEN MATCHED condition to prevent an ambiguous column error (#3940)
|
||||
|
||||
- due to [`667f7d9`](https://github.com/tobymao/sqlglot/commit/667f7d9e94e14ff619998d2001b6116d363f2a1f) - attach INTERPOLATE expressions to WithFill *(PR [#3944](https://github.com/tobymao/sqlglot/pull/3944) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
attach INTERPOLATE expressions to WithFill (#3944)
|
||||
|
||||
- due to [`145fdbf`](https://github.com/tobymao/sqlglot/commit/145fdbf6bb02fa1c55087bfd9f6b3a15fbd4b684) - Redshift date format *(PR [#3942](https://github.com/tobymao/sqlglot/pull/3942) by [@erindru](https://github.com/erindru))*:
|
||||
|
||||
Redshift date format (#3942)
|
||||
|
||||
- due to [`a84a21a`](https://github.com/tobymao/sqlglot/commit/a84a21aaef0e65754e67ecebdfcbf7136c77acc7) - Add timezone support to exp.TimeStrToTime *(PR [#3938](https://github.com/tobymao/sqlglot/pull/3938) by [@erindru](https://github.com/erindru))*:
|
||||
|
||||
Add timezone support to exp.TimeStrToTime (#3938)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`a84a21a`](https://github.com/tobymao/sqlglot/commit/a84a21aaef0e65754e67ecebdfcbf7136c77acc7) - Add timezone support to exp.TimeStrToTime *(PR [#3938](https://github.com/tobymao/sqlglot/pull/3938) by [@erindru](https://github.com/erindru))*
|
||||
- [`70a052a`](https://github.com/tobymao/sqlglot/commit/70a052a672d0c72a3e53b19316defb01144f2907) - transpile from_iso8601_timestamp from presto/trino to duckdb *(PR [#3956](https://github.com/tobymao/sqlglot/pull/3956) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`f68d155`](https://github.com/tobymao/sqlglot/commit/f68d155c38a79a6527685c37f8de8773ce790bca) - exp.Merge, for Trino and Postgres, dont strip the target alias from then WHEN MATCHED condition to prevent an ambiguous column error *(PR [#3940](https://github.com/tobymao/sqlglot/pull/3940) by [@erindru](https://github.com/erindru))*
|
||||
- [`0458dc0`](https://github.com/tobymao/sqlglot/commit/0458dc0fa1978388336b9fa459b28508d7b40f9e) - **optimizer**: expand alias refs recursive CTE edge case patch *(PR [#3943](https://github.com/tobymao/sqlglot/pull/3943) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`145fdbf`](https://github.com/tobymao/sqlglot/commit/145fdbf6bb02fa1c55087bfd9f6b3a15fbd4b684) - Redshift date format *(PR [#3942](https://github.com/tobymao/sqlglot/pull/3942) by [@erindru](https://github.com/erindru))*
|
||||
- [`6233c2c`](https://github.com/tobymao/sqlglot/commit/6233c2c75ab3a3bc0dfbf28d3fa8adc1be719281) - **parser**: Support sqls with DESCRIBE partition *(PR [#3945](https://github.com/tobymao/sqlglot/pull/3945) by [@gp1105739](https://github.com/gp1105739))*
|
||||
- :arrow_lower_right: *fixes issue [#3941](https://github.com/tobymao/sqlglot/issues/3941) opened by [@gp1105739](https://github.com/gp1105739)*
|
||||
- [`85cd6e5`](https://github.com/tobymao/sqlglot/commit/85cd6e507b73be89d2d9b2c88c7370a14b813b5c) - **bigquery**: Map %e to %-d *(PR [#3946](https://github.com/tobymao/sqlglot/pull/3946) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- [`1ba0f03`](https://github.com/tobymao/sqlglot/commit/1ba0f03fbfe5dadc3411c7ff26e6dfbef852491a) - **duckdb**: TIME does not support modifiers *(PR [#3947](https://github.com/tobymao/sqlglot/pull/3947) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`d5d3615`](https://github.com/tobymao/sqlglot/commit/d5d361571cd463869e2243d257f9b6ad0615c070) - **optimizer**: convert TsOrDsToDate to Cast more conservatively *(PR [#3949](https://github.com/tobymao/sqlglot/pull/3949) by [@barakalon](https://github.com/barakalon))*
|
||||
- [`fb6edc7`](https://github.com/tobymao/sqlglot/commit/fb6edc774539704b48e7d2805ef3211636af18aa) - oracle/snowflake comments closes [#3950](https://github.com/tobymao/sqlglot/pull/3950) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`1284fd0`](https://github.com/tobymao/sqlglot/commit/1284fd0a64890d3548af7ed0a0cc05bb6166ccb2) - **oracle**: Revert NVL() being parsed into exp.Anonymous *(PR [#3954](https://github.com/tobymao/sqlglot/pull/3954) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *fixes issue [#3952](https://github.com/tobymao/sqlglot/issues/3952) opened by [@sleshJdev](https://github.com/sleshJdev)*
|
||||
- [`c99f8d5`](https://github.com/tobymao/sqlglot/commit/c99f8d5bda79f16fb0d71ae73127cc826860e104) - **duckdb**: Fix exp.Unnest generation for BQ's nested arrays *(PR [#3931](https://github.com/tobymao/sqlglot/pull/3931) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`f16b0e7`](https://github.com/tobymao/sqlglot/commit/f16b0e7203ad60f0ce50861c4d78176ca53eb2cf) - iteratively generate binary expressions *(PR [#3926](https://github.com/tobymao/sqlglot/pull/3926) by [@MatMoore](https://github.com/MatMoore))*
|
||||
- [`667f7d9`](https://github.com/tobymao/sqlglot/commit/667f7d9e94e14ff619998d2001b6116d363f2a1f) - **clickhouse**: attach INTERPOLATE expressions to WithFill *(PR [#3944](https://github.com/tobymao/sqlglot/pull/3944) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :wrench: Chores
|
||||
- [`c697357`](https://github.com/tobymao/sqlglot/commit/c6973572dfd953b5539bb4e9dcba402c0c3c6acf) - slightly refactor Generator.binary, add stress test *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`6a5f619`](https://github.com/tobymao/sqlglot/commit/6a5f6199f6da0053fa4564e71a17e3b9f91f0496) - New doc - Onboarding Doc *(PR [#3902](https://github.com/tobymao/sqlglot/pull/3902) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
|
||||
|
||||
## [v25.15.0] - 2024-08-19
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`a668655`](https://github.com/tobymao/sqlglot/commit/a668655440815605a566c52b65b28decdfb551eb) - preserve SYSDATE *(PR [#3935](https://github.com/tobymao/sqlglot/pull/3935) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
preserve SYSDATE (#3935)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`be11f4c`](https://github.com/tobymao/sqlglot/commit/be11f4c57c7842f69950bafc3225fb9c139af014) - **clickhouse**: add support for "@"-style parameters *(PR [#3939](https://github.com/tobymao/sqlglot/pull/3939) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`a668655`](https://github.com/tobymao/sqlglot/commit/a668655440815605a566c52b65b28decdfb551eb) - **oracle**: preserve SYSDATE *(PR [#3935](https://github.com/tobymao/sqlglot/pull/3935) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3934](https://github.com/tobymao/sqlglot/issues/3934) opened by [@Hal-H2Apps](https://github.com/Hal-H2Apps)*
|
||||
- [`b824f8a`](https://github.com/tobymao/sqlglot/commit/b824f8a4148ace01750db301daf4a663dc03b580) - **parser**: allow complex expressions for UNPIVOT alias *(PR [#3937](https://github.com/tobymao/sqlglot/pull/3937) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3936](https://github.com/tobymao/sqlglot/issues/3936) opened by [@dbittenbender](https://github.com/dbittenbender)*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`f4c34d3`](https://github.com/tobymao/sqlglot/commit/f4c34d37c5773c37a13437c7e0e7eb27b4e98877) - move "MINUS": TokenType.EXCEPT to hive instead of spark *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.14.0] - 2024-08-19
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`605f1b2`](https://github.com/tobymao/sqlglot/commit/605f1b217d5d1de654cfe2fa1b51435a1a71ae62) - use creatable kind mapping dict for schema<-->database substitution *(PR [#3924](https://github.com/tobymao/sqlglot/pull/3924) by [@treysp](https://github.com/treysp))*:
|
||||
|
||||
use creatable kind mapping dict for schema<-->database substitution (#3924)
|
||||
|
||||
- due to [`f418caa`](https://github.com/tobymao/sqlglot/commit/f418caafa8ed317f9e360c6c8f01bdac596258e5) - skip nullable comparison in is_type by default *(PR [#3927](https://github.com/tobymao/sqlglot/pull/3927) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
skip nullable comparison in is_type by default (#3927)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`f418caa`](https://github.com/tobymao/sqlglot/commit/f418caafa8ed317f9e360c6c8f01bdac596258e5) - skip nullable comparison in is_type by default *(PR [#3927](https://github.com/tobymao/sqlglot/pull/3927) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`605f1b2`](https://github.com/tobymao/sqlglot/commit/605f1b217d5d1de654cfe2fa1b51435a1a71ae62) - **clickhouse**: use creatable kind mapping dict for schema<-->database substitution *(PR [#3924](https://github.com/tobymao/sqlglot/pull/3924) by [@treysp](https://github.com/treysp))*
|
||||
|
||||
|
||||
## [v25.13.0] - 2024-08-17
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`102f5d4`](https://github.com/tobymao/sqlglot/commit/102f5d48279ac1a7a1851737f55a13bd08512f3d) - infer set op types more accurately *(PR [#3918](https://github.com/tobymao/sqlglot/pull/3918) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
infer set op types more accurately (#3918)
|
||||
|
||||
- due to [`46496a6`](https://github.com/tobymao/sqlglot/commit/46496a6af80bd49d36ef8d265800679d2b07c4db) - improve transpilation of nullable/non-nullable data types *(PR [#3921](https://github.com/tobymao/sqlglot/pull/3921) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
improve transpilation of nullable/non-nullable data types (#3921)
|
||||
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`c74a8fd`](https://github.com/tobymao/sqlglot/commit/c74a8fd2acd859f5947f27a8f091f13fba1d39e4) - **clickhouse**: make try_cast toXXXOrNull() functions case-specific *(PR [#3917](https://github.com/tobymao/sqlglot/pull/3917) by [@treysp](https://github.com/treysp))*
|
||||
- [`102f5d4`](https://github.com/tobymao/sqlglot/commit/102f5d48279ac1a7a1851737f55a13bd08512f3d) - **optimizer**: infer set op types more accurately *(PR [#3918](https://github.com/tobymao/sqlglot/pull/3918) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3916](https://github.com/tobymao/sqlglot/issues/3916) opened by [@racevedoo](https://github.com/racevedoo)*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`1d436d4`](https://github.com/tobymao/sqlglot/commit/1d436d45b4469bb8195dd3597319b6fc5c3f2344) - **clickhouse**: transpile TRY_CAST(x AS T) to CAST(x AS Nullable(T)) *(PR [#3919](https://github.com/tobymao/sqlglot/pull/3919) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`46496a6`](https://github.com/tobymao/sqlglot/commit/46496a6af80bd49d36ef8d265800679d2b07c4db) - **clickhouse**: improve transpilation of nullable/non-nullable data types *(PR [#3921](https://github.com/tobymao/sqlglot/pull/3921) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.12.0] - 2024-08-15
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`e8e70f3`](https://github.com/tobymao/sqlglot/commit/e8e70f3a6cc2ca24de2afe622bbcbccb1ac8aeb3) - treat DATABASE kind as SCHEMA (and conversely) in exp.Create *(PR [#3912](https://github.com/tobymao/sqlglot/pull/3912) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
treat DATABASE kind as SCHEMA (and conversely) in exp.Create (#3912)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`9a66903`](https://github.com/tobymao/sqlglot/commit/9a66903975f16a09d84337a8405bf70945706412) - **clickhouse**: add support for TryCast generation *(PR [#3913](https://github.com/tobymao/sqlglot/pull/3913) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`7965cac`](https://github.com/tobymao/sqlglot/commit/7965cace1d9632c865cae257781072b0932b709d) - **clickhouse**: wrap query in CTAS when COMMENT prop is present *(PR [#3911](https://github.com/tobymao/sqlglot/pull/3911) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`e8e70f3`](https://github.com/tobymao/sqlglot/commit/e8e70f3a6cc2ca24de2afe622bbcbccb1ac8aeb3) - **clickhouse**: treat DATABASE kind as SCHEMA (and conversely) in exp.Create *(PR [#3912](https://github.com/tobymao/sqlglot/pull/3912) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.11.3] - 2024-08-14
|
||||
### :bug: Bug Fixes
|
||||
- [`57f7aa9`](https://github.com/tobymao/sqlglot/commit/57f7aa9108ed38c0e83ef5bf4fac900434fac777) - **clickhouse**: COMMENT property in CTAS needs to come last *(PR [#3910](https://github.com/tobymao/sqlglot/pull/3910) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.11.2] - 2024-08-14
|
||||
### :bug: Bug Fixes
|
||||
- [`c22f411`](https://github.com/tobymao/sqlglot/commit/c22f41129985ecfd3b3906b9594ca1692b91708c) - **clickhouse**: ensure we generate the Table in creatable_sql if it represents a db ref *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`19eee93`](https://github.com/tobymao/sqlglot/commit/19eee93c8027e6c612611d3b54980e193e0b6f49) - various fixups for unnest(generatedatearray) transpilation *(PR [#3906](https://github.com/tobymao/sqlglot/pull/3906) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.11.1] - 2024-08-13
|
||||
### :sparkles: New Features
|
||||
- [`790c1b1`](https://github.com/tobymao/sqlglot/commit/790c1b141d4bc2206df017c70416b589932886a4) - **clickhouse**: support PARTITION BY, SETTINGS in Insert expression *(PR [#3904](https://github.com/tobymao/sqlglot/pull/3904) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.11.0] - 2024-08-13
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`0428c37`](https://github.com/tobymao/sqlglot/commit/0428c37e11f42be8eba352e69c1d2e7425824d38) - Support ALTER VIEW AS SELECT *(PR [#3873](https://github.com/tobymao/sqlglot/pull/3873) by [@xiaohui-sun](https://github.com/xiaohui-sun))*:
|
||||
|
||||
Support ALTER VIEW AS SELECT (#3873)
|
||||
|
||||
- due to [`a666117`](https://github.com/tobymao/sqlglot/commit/a666117dcb887031f5995c50d687405b9c145fbd) - parse v NOT IN (subquery) as v <> ALL (subquery) *(PR [#3891](https://github.com/tobymao/sqlglot/pull/3891) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
parse v NOT IN (subquery) as v <> ALL (subquery) (#3891)
|
||||
|
||||
- due to [`d968932`](https://github.com/tobymao/sqlglot/commit/d968932ef742e97ccf3ec6cdca0bc3319830f0a9) - treat identifiers as case-sensitive, handle EMPTY table property, generate DateStrToDate *(PR [#3895](https://github.com/tobymao/sqlglot/pull/3895) by [@jwhitaker-gridcog](https://github.com/jwhitaker-gridcog))*:
|
||||
|
||||
treat identifiers as case-sensitive, handle EMPTY table property, generate DateStrToDate (#3895)
|
||||
|
||||
- due to [`1d7319a`](https://github.com/tobymao/sqlglot/commit/1d7319a8425aace6c11f59552fdd19bdbf5efd03) - transpile Unnest(GenerateDateArray(...)) to various dialects *(PR [#3899](https://github.com/tobymao/sqlglot/pull/3899) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
transpile Unnest(GenerateDateArray(...)) to various dialects (#3899)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`0428c37`](https://github.com/tobymao/sqlglot/commit/0428c37e11f42be8eba352e69c1d2e7425824d38) - **parser**: Support ALTER VIEW AS SELECT *(PR [#3873](https://github.com/tobymao/sqlglot/pull/3873) by [@xiaohui-sun](https://github.com/xiaohui-sun))*
|
||||
- [`8a48458`](https://github.com/tobymao/sqlglot/commit/8a48458e20e6d0833638e750565da138bdcd5d55) - **athena**: parse UNLOAD into exp.Command closes [#3896](https://github.com/tobymao/sqlglot/pull/3896) *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`6f1527f`](https://github.com/tobymao/sqlglot/commit/6f1527fd3ebd16f49edb351f050a1db687824530) - **bigquery**: transpile format_datetime, datetime_trunc to duckdb *(PR [#3894](https://github.com/tobymao/sqlglot/pull/3894) by [@skadel](https://github.com/skadel))*
|
||||
- [`1d7319a`](https://github.com/tobymao/sqlglot/commit/1d7319a8425aace6c11f59552fdd19bdbf5efd03) - transpile Unnest(GenerateDateArray(...)) to various dialects *(PR [#3899](https://github.com/tobymao/sqlglot/pull/3899) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`2cac14f`](https://github.com/tobymao/sqlglot/commit/2cac14f480dcaf458b1eb36b694770ce24f56e61) - generate set ops in ALTER VIEW AS statement *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`a666117`](https://github.com/tobymao/sqlglot/commit/a666117dcb887031f5995c50d687405b9c145fbd) - **snowflake**: parse v NOT IN (subquery) as v <> ALL (subquery) *(PR [#3891](https://github.com/tobymao/sqlglot/pull/3891) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3890](https://github.com/tobymao/sqlglot/issues/3890) opened by [@ajuszczak](https://github.com/ajuszczak)*
|
||||
- [`924a4af`](https://github.com/tobymao/sqlglot/commit/924a4af146952e84688fdccb7b63883fcd7fb255) - **oracle**: preserve function-style MOD syntax fixes [#3897](https://github.com/tobymao/sqlglot/pull/3897) *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`d968932`](https://github.com/tobymao/sqlglot/commit/d968932ef742e97ccf3ec6cdca0bc3319830f0a9) - **clickhouse**: treat identifiers as case-sensitive, handle EMPTY table property, generate DateStrToDate *(PR [#3895](https://github.com/tobymao/sqlglot/pull/3895) by [@jwhitaker-gridcog](https://github.com/jwhitaker-gridcog))*
|
||||
- [`3e5e730`](https://github.com/tobymao/sqlglot/commit/3e5e7300ec184024f871669db48d68476b3fa4df) - **clickhouse**: generate exp.Values correctly, handle `FORMAT Values` *(PR [#3900](https://github.com/tobymao/sqlglot/pull/3900) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`bea3c08`](https://github.com/tobymao/sqlglot/commit/bea3c08e46a020d8545b702c77f0db18c99f1c55) - **parser**: improve performance of OUTER/CROSS APPLY parsing *(PR [#3901](https://github.com/tobymao/sqlglot/pull/3901) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3898](https://github.com/tobymao/sqlglot/issues/3898) opened by [@ewhitley](https://github.com/ewhitley)*
|
||||
|
||||
|
||||
## [v25.10.0] - 2024-08-08
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`3eb46db`](https://github.com/tobymao/sqlglot/commit/3eb46db5c429f50b5bb6c0c5517a5f7c1084b5ea) - switch off CSV file schema inference by default *(PR [#3879](https://github.com/tobymao/sqlglot/pull/3879) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
switch off CSV file schema inference by default (#3879)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`3e4fcf7`](https://github.com/tobymao/sqlglot/commit/3e4fcf7e8f6a322c14470de6c5dbba152bc9b2fe) - **databricks**: Add support for STREAMING tables *(PR [#3878](https://github.com/tobymao/sqlglot/pull/3878) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *addresses issue [#3876](https://github.com/tobymao/sqlglot/issues/3876) opened by [@ericvergnaud](https://github.com/ericvergnaud)*
|
||||
- [`528f690`](https://github.com/tobymao/sqlglot/commit/528f6908001db2f132edfa3c61c21815f7e9dc2f) - **duckdb**: Transpile Snowflake's CONVERT_TIMEZONE 3-arg version *(PR [#3883](https://github.com/tobymao/sqlglot/pull/3883) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *addresses issue [#3875](https://github.com/tobymao/sqlglot/issues/3875) opened by [@milonimrod](https://github.com/milonimrod)*
|
||||
- [`411f62a`](https://github.com/tobymao/sqlglot/commit/411f62ad27f8cbe0d9a429e0cafdf4bd9eb2749f) - **bigquery**: Support for GENERATE_TIMESTAMP_ARRAY, DDB transpilation *(PR [#3888](https://github.com/tobymao/sqlglot/pull/3888) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`7169e6e`](https://github.com/tobymao/sqlglot/commit/7169e6ef52d24754059b9ee4324398d22ddff0da) - **bigquery**: ensure Funcs are preserved when used as Tables *(PR [#3877](https://github.com/tobymao/sqlglot/pull/3877) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`62ceed2`](https://github.com/tobymao/sqlglot/commit/62ceed2fa3cd7b41919839d837b860f3814fa769) - **redshift**: parse first arg in DATE_PART into a Var fixes [#3882](https://github.com/tobymao/sqlglot/pull/3882) *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`2ad9bfe`](https://github.com/tobymao/sqlglot/commit/2ad9bfef71ae707b83f604f16b47aa583d082c3b) - **snowflake**: support table qualification in USING clause *(PR [#3885](https://github.com/tobymao/sqlglot/pull/3885) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3881](https://github.com/tobymao/sqlglot/issues/3881) opened by [@dlahyani](https://github.com/dlahyani)*
|
||||
- [`ef16b1d`](https://github.com/tobymao/sqlglot/commit/ef16b1da6b43647a0ca08d69eaf3610e3b72671f) - Fix COLLATE's RHS parsing *(PR [#3887](https://github.com/tobymao/sqlglot/pull/3887) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *fixes issue [#3880](https://github.com/tobymao/sqlglot/issues/3880) opened by [@ewhitley](https://github.com/ewhitley)*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`3eb46db`](https://github.com/tobymao/sqlglot/commit/3eb46db5c429f50b5bb6c0c5517a5f7c1084b5ea) - **optimizer**: switch off CSV file schema inference by default *(PR [#3879](https://github.com/tobymao/sqlglot/pull/3879) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.9.0] - 2024-08-05
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`64e187c`](https://github.com/tobymao/sqlglot/commit/64e187c52cd9725ba79e6afbd444382eba9e5827) - transpile postgres impliclitly exploding GENERATE_SERIES proje… *(PR [#3853](https://github.com/tobymao/sqlglot/pull/3853) by [@georgesittas](https://github.com/georgesittas))*:
|
||||
|
||||
transpile postgres impliclitly exploding GENERATE_SERIES proje… (#3853)
|
||||
|
||||
- due to [`e53e7cc`](https://github.com/tobymao/sqlglot/commit/e53e7cc02a224563d0a61b0a39298d606b9bac80) - Generation of exp.ArrayConcat for 2-arg based dialects *(PR [#3864](https://github.com/tobymao/sqlglot/pull/3864) by [@VaggelisD](https://github.com/VaggelisD))*:
|
||||
|
||||
Generation of exp.ArrayConcat for 2-arg based dialects (#3864)
|
||||
|
||||
- due to [`659b8bf`](https://github.com/tobymao/sqlglot/commit/659b8bf12e396856d1562ee4678b4f687629e081) - Support for BQ's exp.GenerateDateArray generation *(PR [#3865](https://github.com/tobymao/sqlglot/pull/3865) by [@VaggelisD](https://github.com/VaggelisD))*:
|
||||
|
||||
Support for BQ's exp.GenerateDateArray generation (#3865)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`6afed2a`](https://github.com/tobymao/sqlglot/commit/6afed2aecc0ce186ff6c484b1ad32ac6a2fb61bc) - **duckdb**: Support for exp.TimeDiff generation *(PR [#3856](https://github.com/tobymao/sqlglot/pull/3856) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- [`64e187c`](https://github.com/tobymao/sqlglot/commit/64e187c52cd9725ba79e6afbd444382eba9e5827) - transpile postgres impliclitly exploding GENERATE_SERIES proje… *(PR [#3853](https://github.com/tobymao/sqlglot/pull/3853) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *addresses issue [#3818](https://github.com/tobymao/sqlglot/issues/3818) opened by [@wojciechowski-p](https://github.com/wojciechowski-p)*
|
||||
- [`8a948c8`](https://github.com/tobymao/sqlglot/commit/8a948c805f7534e266557e1aa08bee0982340685) - **teradata**: Parse RENAME TABLE as Command *(PR [#3863](https://github.com/tobymao/sqlglot/pull/3863) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *addresses issue [#3861](https://github.com/tobymao/sqlglot/issues/3861) opened by [@EdouardW](https://github.com/EdouardW)*
|
||||
- [`659b8bf`](https://github.com/tobymao/sqlglot/commit/659b8bf12e396856d1562ee4678b4f687629e081) - **duckdb**: Support for BQ's exp.GenerateDateArray generation *(PR [#3865](https://github.com/tobymao/sqlglot/pull/3865) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- [`734f54b`](https://github.com/tobymao/sqlglot/commit/734f54bb6ec697a5213f046fbb1e8174b2c31115) - **snowflake**: add support for a a couple of missing clauses in PIVOT clause *(PR [#3867](https://github.com/tobymao/sqlglot/pull/3867) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`8710763`](https://github.com/tobymao/sqlglot/commit/87107631378b0972115a01cc0bb99dbfc44a66d7) - **presto**: map %W to %A in the TIME_MAPPING *(PR [#3855](https://github.com/tobymao/sqlglot/pull/3855) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- :arrow_lower_right: *fixes issue [#3854](https://github.com/tobymao/sqlglot/issues/3854) opened by [@ddelzell](https://github.com/ddelzell)*
|
||||
- [`532f3c8`](https://github.com/tobymao/sqlglot/commit/532f3c8714220058170790b13977cc66760841dc) - **duckdb**: Add implicit casts to DATE_DIFF *(PR [#3857](https://github.com/tobymao/sqlglot/pull/3857) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- [`299c4a5`](https://github.com/tobymao/sqlglot/commit/299c4a559dd04047d5a4c4691f8965972842fe7d) - **clickhouse**: Fix SETTINGS parsing *(PR [#3859](https://github.com/tobymao/sqlglot/pull/3859) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *fixes issue [#3858](https://github.com/tobymao/sqlglot/issues/3858) opened by [@obazna](https://github.com/obazna)*
|
||||
- [`810d23d`](https://github.com/tobymao/sqlglot/commit/810d23d4e42f9a7de83015ec425dff9223598219) - **parser**: make assignment parsing more lenient by allowing keyword in LHS *(PR [#3866](https://github.com/tobymao/sqlglot/pull/3866) by [@georgesittas](https://github.com/georgesittas))*
|
||||
- [`e53e7cc`](https://github.com/tobymao/sqlglot/commit/e53e7cc02a224563d0a61b0a39298d606b9bac80) - Generation of exp.ArrayConcat for 2-arg based dialects *(PR [#3864](https://github.com/tobymao/sqlglot/pull/3864) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- [`813f127`](https://github.com/tobymao/sqlglot/commit/813f127b293e7087d174f3f632b65ba7b24bc9e3) - **duckdb**: Allow DESCRIBE as a _parse_select() path *(PR [#3871](https://github.com/tobymao/sqlglot/pull/3871) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *fixes issue [#3869](https://github.com/tobymao/sqlglot/issues/3869) opened by [@cpcloud](https://github.com/cpcloud)*
|
||||
- [`6ff0c01`](https://github.com/tobymao/sqlglot/commit/6ff0c01a5b8b19e3090b8cf08aabbb4b27425abb) - Fixed size array parsing *(PR [#3870](https://github.com/tobymao/sqlglot/pull/3870) by [@VaggelisD](https://github.com/VaggelisD))*
|
||||
- :arrow_lower_right: *fixes issue [#3868](https://github.com/tobymao/sqlglot/issues/3868) opened by [@tekumara](https://github.com/tekumara)*
|
||||
|
||||
|
||||
## [v25.8.1] - 2024-07-30
|
||||
### :bug: Bug Fixes
|
||||
- [`a295b3a`](https://github.com/tobymao/sqlglot/commit/a295b3adbef0eff0b3f6c3b8b97b1eaa8c13f144) - **tsql**: regression related to CTEs in CREATE VIEW AS statements *(PR [#3852](https://github.com/tobymao/sqlglot/pull/3852) by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
||||
|
||||
## [v25.8.0] - 2024-07-29
|
||||
### :sparkles: New Features
|
||||
- [`e37d63a`](https://github.com/tobymao/sqlglot/commit/e37d63a17d4709135c1de7876b2898cf7bd2e641) - **bigquery**: add support for BYTEINT closes [#3838](https://github.com/tobymao/sqlglot/pull/3838) *(commit by [@georgesittas](https://github.com/georgesittas))*
|
||||
|
@ -4238,3 +4479,15 @@ Changelog
|
|||
[v25.7.0]: https://github.com/tobymao/sqlglot/compare/v25.6.1...v25.7.0
|
||||
[v25.7.1]: https://github.com/tobymao/sqlglot/compare/v25.7.0...v25.7.1
|
||||
[v25.8.0]: https://github.com/tobymao/sqlglot/compare/v25.7.1...v25.8.0
|
||||
[v25.8.1]: https://github.com/tobymao/sqlglot/compare/v25.8.0...v25.8.1
|
||||
[v25.9.0]: https://github.com/tobymao/sqlglot/compare/v25.8.1...v25.9.0
|
||||
[v25.10.0]: https://github.com/tobymao/sqlglot/compare/v25.9.0...v25.10.0
|
||||
[v25.11.0]: https://github.com/tobymao/sqlglot/compare/v25.10.0...v25.11.0
|
||||
[v25.11.1]: https://github.com/tobymao/sqlglot/compare/v25.11.0...v25.11.1
|
||||
[v25.11.2]: https://github.com/tobymao/sqlglot/compare/v25.11.1...v25.11.2
|
||||
[v25.11.3]: https://github.com/tobymao/sqlglot/compare/v25.11.2...v25.11.3
|
||||
[v25.12.0]: https://github.com/tobymao/sqlglot/compare/v25.11.3...v25.12.0
|
||||
[v25.13.0]: https://github.com/tobymao/sqlglot/compare/v25.12.0...v25.13.0
|
||||
[v25.14.0]: https://github.com/tobymao/sqlglot/compare/v25.13.0...v25.14.0
|
||||
[v25.15.0]: https://github.com/tobymao/sqlglot/compare/v25.14.0...v25.15.0
|
||||
[v25.16.0]: https://github.com/tobymao/sqlglot/compare/v25.15.0...v25.16.0
|
||||
|
|
|
@ -10,7 +10,7 @@ Syntax [errors](#parser-errors) are highlighted and dialect incompatibilities ca
|
|||
|
||||
Learn more about SQLGlot in the API [documentation](https://sqlglot.com/) and the expression tree [primer](https://github.com/tobymao/sqlglot/blob/main/posts/ast_primer.md).
|
||||
|
||||
Contributions are very welcome in SQLGlot; read the [contribution guide](https://github.com/tobymao/sqlglot/blob/main/CONTRIBUTING.md) to get started!
|
||||
Contributions are very welcome in SQLGlot; read the [contribution guide](https://github.com/tobymao/sqlglot/blob/main/CONTRIBUTING.md) and the [onboarding document](https://github.com/tobymao/sqlglot/blob/main/posts/ast_primer.md) to get started!
|
||||
|
||||
## Table of Contents
|
||||
|
||||
|
|
File diff suppressed because one or more lines are too long
|
@ -76,8 +76,8 @@
|
|||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'25.8.0'</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">25</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'25.16.0'</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">25</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -97,7 +97,7 @@
|
|||
<section id="version">
|
||||
<div class="attr variable">
|
||||
<span class="name">version</span><span class="annotation">: str</span> =
|
||||
<span class="default_value">'25.8.0'</span>
|
||||
<span class="default_value">'25.16.0'</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
@ -109,7 +109,7 @@
|
|||
<section id="version_tuple">
|
||||
<div class="attr variable">
|
||||
<span class="name">version_tuple</span><span class="annotation">: object</span> =
|
||||
<span class="default_value">(25, 8, 0)</span>
|
||||
<span class="default_value">(25, 16, 0)</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
@ -48,6 +48,9 @@
|
|||
<li>
|
||||
<a class="variable" href="#RisingWave.Generator.SUPPORTS_UESCAPE">SUPPORTS_UESCAPE</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="variable" href="#RisingWave.Generator.SUPPORTS_NULLABLE_TYPES">SUPPORTS_NULLABLE_TYPES</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="variable" href="#RisingWave.Generator.AFTER_HAVING_MODIFIER_TRANSFORMS">AFTER_HAVING_MODIFIER_TRANSFORMS</a>
|
||||
</li>
|
||||
|
@ -87,6 +90,9 @@
|
|||
<li>
|
||||
<a class="variable" href="#RisingWave.INVERSE_FORMAT_TRIE">INVERSE_FORMAT_TRIE</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="variable" href="#RisingWave.INVERSE_CREATABLE_KIND_MAPPING">INVERSE_CREATABLE_KIND_MAPPING</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="variable" href="#RisingWave.ESCAPED_SEQUENCES">ESCAPED_SEQUENCES</a>
|
||||
</li>
|
||||
|
@ -318,6 +324,18 @@
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="RisingWave.INVERSE_CREATABLE_KIND_MAPPING" class="classattr">
|
||||
<div class="attr variable">
|
||||
<span class="name">INVERSE_CREATABLE_KIND_MAPPING</span><span class="annotation">: dict[str, str]</span> =
|
||||
<span class="default_value">{}</span>
|
||||
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#RisingWave.INVERSE_CREATABLE_KIND_MAPPING"></a>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="RisingWave.ESCAPED_SEQUENCES" class="classattr">
|
||||
<div class="attr variable">
|
||||
|
@ -503,6 +521,9 @@
|
|||
<dd id="RisingWave.FORCE_EARLY_ALIAS_REF_EXPANSION" class="variable"><a href="dialect.html#Dialect.FORCE_EARLY_ALIAS_REF_EXPANSION">FORCE_EARLY_ALIAS_REF_EXPANSION</a></dd>
|
||||
<dd id="RisingWave.EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY" class="variable"><a href="dialect.html#Dialect.EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY">EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY</a></dd>
|
||||
<dd id="RisingWave.SUPPORTS_ORDER_BY_ALL" class="variable"><a href="dialect.html#Dialect.SUPPORTS_ORDER_BY_ALL">SUPPORTS_ORDER_BY_ALL</a></dd>
|
||||
<dd id="RisingWave.HAS_DISTINCT_ARRAY_CONSTRUCTORS" class="variable"><a href="dialect.html#Dialect.HAS_DISTINCT_ARRAY_CONSTRUCTORS">HAS_DISTINCT_ARRAY_CONSTRUCTORS</a></dd>
|
||||
<dd id="RisingWave.SUPPORTS_FIXED_SIZE_ARRAYS" class="variable"><a href="dialect.html#Dialect.SUPPORTS_FIXED_SIZE_ARRAYS">SUPPORTS_FIXED_SIZE_ARRAYS</a></dd>
|
||||
<dd id="RisingWave.CREATABLE_KIND_MAPPING" class="variable"><a href="dialect.html#Dialect.CREATABLE_KIND_MAPPING">CREATABLE_KIND_MAPPING</a></dd>
|
||||
<dd id="RisingWave.DATE_PART_MAPPING" class="variable"><a href="dialect.html#Dialect.DATE_PART_MAPPING">DATE_PART_MAPPING</a></dd>
|
||||
<dd id="RisingWave.TYPE_TO_EXPRESSIONS" class="variable"><a href="dialect.html#Dialect.TYPE_TO_EXPRESSIONS">TYPE_TO_EXPRESSIONS</a></dd>
|
||||
<dd id="RisingWave.ANNOTATORS" class="variable"><a href="dialect.html#Dialect.ANNOTATORS">ANNOTATORS</a></dd>
|
||||
|
@ -644,6 +665,18 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="RisingWave.Generator.SUPPORTS_NULLABLE_TYPES" class="classattr">
|
||||
<div class="attr variable">
|
||||
<span class="name">SUPPORTS_NULLABLE_TYPES</span> =
|
||||
<span class="default_value">False</span>
|
||||
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#RisingWave.Generator.SUPPORTS_NULLABLE_TYPES"></a>
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="RisingWave.Generator.AFTER_HAVING_MODIFIER_TRANSFORMS" class="classattr">
|
||||
<div class="attr variable">
|
||||
|
@ -712,6 +745,8 @@ Default: True</li>
|
|||
<dd id="RisingWave.Generator.WITH_PROPERTIES_PREFIX" class="variable"><a href="../generator.html#Generator.WITH_PROPERTIES_PREFIX">WITH_PROPERTIES_PREFIX</a></dd>
|
||||
<dd id="RisingWave.Generator.QUOTE_JSON_PATH" class="variable"><a href="../generator.html#Generator.QUOTE_JSON_PATH">QUOTE_JSON_PATH</a></dd>
|
||||
<dd id="RisingWave.Generator.PAD_FILL_PATTERN_IS_REQUIRED" class="variable"><a href="../generator.html#Generator.PAD_FILL_PATTERN_IS_REQUIRED">PAD_FILL_PATTERN_IS_REQUIRED</a></dd>
|
||||
<dd id="RisingWave.Generator.SUPPORTS_EXPLODING_PROJECTIONS" class="variable"><a href="../generator.html#Generator.SUPPORTS_EXPLODING_PROJECTIONS">SUPPORTS_EXPLODING_PROJECTIONS</a></dd>
|
||||
<dd id="RisingWave.Generator.SUPPORTS_CONVERT_TIMEZONE" class="variable"><a href="../generator.html#Generator.SUPPORTS_CONVERT_TIMEZONE">SUPPORTS_CONVERT_TIMEZONE</a></dd>
|
||||
<dd id="RisingWave.Generator.PARSE_JSON_NAME" class="variable"><a href="../generator.html#Generator.PARSE_JSON_NAME">PARSE_JSON_NAME</a></dd>
|
||||
<dd id="RisingWave.Generator.TIME_PART_SINGULARS" class="variable"><a href="../generator.html#Generator.TIME_PART_SINGULARS">TIME_PART_SINGULARS</a></dd>
|
||||
<dd id="RisingWave.Generator.TOKEN_MAPPING" class="variable"><a href="../generator.html#Generator.TOKEN_MAPPING">TOKEN_MAPPING</a></dd>
|
||||
|
@ -963,7 +998,7 @@ Default: True</li>
|
|||
<dd id="RisingWave.Generator.altersortkey_sql" class="function"><a href="../generator.html#Generator.altersortkey_sql">altersortkey_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.renametable_sql" class="function"><a href="../generator.html#Generator.renametable_sql">renametable_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.renamecolumn_sql" class="function"><a href="../generator.html#Generator.renamecolumn_sql">renamecolumn_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.altertable_sql" class="function"><a href="../generator.html#Generator.altertable_sql">altertable_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.alter_sql" class="function"><a href="../generator.html#Generator.alter_sql">alter_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.add_column_sql" class="function"><a href="../generator.html#Generator.add_column_sql">add_column_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.droppartition_sql" class="function"><a href="../generator.html#Generator.droppartition_sql">droppartition_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.addconstraint_sql" class="function"><a href="../generator.html#Generator.addconstraint_sql">addconstraint_sql</a></dd>
|
||||
|
@ -1037,7 +1072,6 @@ Default: True</li>
|
|||
<dd id="RisingWave.Generator.predict_sql" class="function"><a href="../generator.html#Generator.predict_sql">predict_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.forin_sql" class="function"><a href="../generator.html#Generator.forin_sql">forin_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.refresh_sql" class="function"><a href="../generator.html#Generator.refresh_sql">refresh_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.operator_sql" class="function"><a href="../generator.html#Generator.operator_sql">operator_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.toarray_sql" class="function"><a href="../generator.html#Generator.toarray_sql">toarray_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.tsordstotime_sql" class="function"><a href="../generator.html#Generator.tsordstotime_sql">tsordstotime_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.tsordstotimestamp_sql" class="function"><a href="../generator.html#Generator.tsordstotimestamp_sql">tsordstotimestamp_sql</a></dd>
|
||||
|
@ -1064,6 +1098,9 @@ Default: True</li>
|
|||
<dd id="RisingWave.Generator.changes_sql" class="function"><a href="../generator.html#Generator.changes_sql">changes_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.pad_sql" class="function"><a href="../generator.html#Generator.pad_sql">pad_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.summarize_sql" class="function"><a href="../generator.html#Generator.summarize_sql">summarize_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.explodinggenerateseries_sql" class="function"><a href="../generator.html#Generator.explodinggenerateseries_sql">explodinggenerateseries_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.arrayconcat_sql" class="function"><a href="../generator.html#Generator.arrayconcat_sql">arrayconcat_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.converttimezone_sql" class="function"><a href="../generator.html#Generator.converttimezone_sql">converttimezone_sql</a></dd>
|
||||
|
||||
</div>
|
||||
<div><dt><a href="postgres.html#Postgres.Generator">sqlglot.dialects.postgres.Postgres.Generator</a></dt>
|
||||
|
@ -1083,6 +1120,7 @@ Default: True</li>
|
|||
<dd id="RisingWave.Generator.MULTI_ARG_DISTINCT" class="variable"><a href="postgres.html#Postgres.Generator.MULTI_ARG_DISTINCT">MULTI_ARG_DISTINCT</a></dd>
|
||||
<dd id="RisingWave.Generator.CAN_IMPLEMENT_ARRAY_ANY" class="variable"><a href="postgres.html#Postgres.Generator.CAN_IMPLEMENT_ARRAY_ANY">CAN_IMPLEMENT_ARRAY_ANY</a></dd>
|
||||
<dd id="RisingWave.Generator.COPY_HAS_INTO_KEYWORD" class="variable"><a href="postgres.html#Postgres.Generator.COPY_HAS_INTO_KEYWORD">COPY_HAS_INTO_KEYWORD</a></dd>
|
||||
<dd id="RisingWave.Generator.ARRAY_CONCAT_IS_VAR_LEN" class="variable"><a href="postgres.html#Postgres.Generator.ARRAY_CONCAT_IS_VAR_LEN">ARRAY_CONCAT_IS_VAR_LEN</a></dd>
|
||||
<dd id="RisingWave.Generator.SUPPORTED_JSON_PATH_PARTS" class="variable"><a href="postgres.html#Postgres.Generator.SUPPORTED_JSON_PATH_PARTS">SUPPORTED_JSON_PATH_PARTS</a></dd>
|
||||
<dd id="RisingWave.Generator.TYPE_MAPPING" class="variable"><a href="postgres.html#Postgres.Generator.TYPE_MAPPING">TYPE_MAPPING</a></dd>
|
||||
<dd id="RisingWave.Generator.TRANSFORMS" class="variable"><a href="postgres.html#Postgres.Generator.TRANSFORMS">TRANSFORMS</a></dd>
|
||||
|
@ -1095,6 +1133,7 @@ Default: True</li>
|
|||
<dd id="RisingWave.Generator.alterset_sql" class="function"><a href="postgres.html#Postgres.Generator.alterset_sql">alterset_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.datatype_sql" class="function"><a href="postgres.html#Postgres.Generator.datatype_sql">datatype_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.cast_sql" class="function"><a href="postgres.html#Postgres.Generator.cast_sql">cast_sql</a></dd>
|
||||
<dd id="RisingWave.Generator.array_sql" class="function"><a href="postgres.html#Postgres.Generator.array_sql">array_sql</a></dd>
|
||||
|
||||
</div>
|
||||
</dl>
|
||||
|
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
@ -408,21 +408,23 @@
|
|||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">"Tables must support the same table args as schema"</span><span class="p">)</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -496,21 +498,23 @@
|
|||
</span><span id="execute-75"><a href="#execute-75"><span class="linenos">75</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">"Tables must support the same table args as schema"</span><span class="p">)</span>
|
||||
</span><span id="execute-76"><a href="#execute-76"><span class="linenos">76</span></a>
|
||||
</span><span id="execute-77"><a href="#execute-77"><span class="linenos">77</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a>
|
||||
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a>
|
||||
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span>
|
||||
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span>
|
||||
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="p">)</span>
|
||||
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a>
|
||||
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="execute-84"><a href="#execute-84"><span class="linenos">84</span></a>
|
||||
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="execute-86"><a href="#execute-86"><span class="linenos">86</span></a>
|
||||
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a>
|
||||
</span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a>
|
||||
</span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-91"><a href="#execute-91"><span class="linenos">91</span></a>
|
||||
</span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-93"><a href="#execute-93"><span class="linenos">93</span></a>
|
||||
</span><span id="execute-94"><a href="#execute-94"><span class="linenos">94</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
@ -1895,7 +1895,7 @@ belong to some totally-ordered set.</p>
|
|||
<section id="DATE_UNITS">
|
||||
<div class="attr variable">
|
||||
<span class="name">DATE_UNITS</span> =
|
||||
<span class="default_value">{'day', 'quarter', 'month', 'year_month', 'week', 'year'}</span>
|
||||
<span class="default_value">{'year', 'quarter', 'week', 'day', 'month', 'year_month'}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
|
@ -391,6 +391,7 @@
|
|||
<dd id="JSONPathTokenizer.HEREDOC_TAG_IS_IDENTIFIER" class="variable"><a href="tokens.html#Tokenizer.HEREDOC_TAG_IS_IDENTIFIER">HEREDOC_TAG_IS_IDENTIFIER</a></dd>
|
||||
<dd id="JSONPathTokenizer.HEREDOC_STRING_ALTERNATIVE" class="variable"><a href="tokens.html#Tokenizer.HEREDOC_STRING_ALTERNATIVE">HEREDOC_STRING_ALTERNATIVE</a></dd>
|
||||
<dd id="JSONPathTokenizer.STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS" class="variable"><a href="tokens.html#Tokenizer.STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS">STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS</a></dd>
|
||||
<dd id="JSONPathTokenizer.NESTED_COMMENTS" class="variable"><a href="tokens.html#Tokenizer.NESTED_COMMENTS">NESTED_COMMENTS</a></dd>
|
||||
<dd id="JSONPathTokenizer.WHITE_SPACE" class="variable"><a href="tokens.html#Tokenizer.WHITE_SPACE">WHITE_SPACE</a></dd>
|
||||
<dd id="JSONPathTokenizer.COMMANDS" class="variable"><a href="tokens.html#Tokenizer.COMMANDS">COMMANDS</a></dd>
|
||||
<dd id="JSONPathTokenizer.COMMAND_PREFIX_TOKENS" class="variable"><a href="tokens.html#Tokenizer.COMMAND_PREFIX_TOKENS">COMMAND_PREFIX_TOKENS</a></dd>
|
||||
|
@ -585,7 +586,7 @@
|
|||
<div class="attr variable">
|
||||
<span class="name">ALL_JSON_PATH_PARTS</span> =
|
||||
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{<class '<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>'>, <class '<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>'>, <class '<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>'>, <class '<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>'>, <class '<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>'>, <class '<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>'>, <class '<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>'>, <class '<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>'>, <class '<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>'>, <class '<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>'>}</span>
|
||||
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{<class '<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>'>, <class '<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>'>, <class '<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>'>, <class '<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>'>, <class '<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>'>, <class '<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>'>, <class '<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>'>, <class '<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>'>, <class '<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>'>, <class '<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>'>}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
File diff suppressed because one or more lines are too long
|
@ -119,140 +119,148 @@
|
|||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">)</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="p">):</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">is_string</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="ow">and</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="p">):</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="p">)</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="p">):</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="p">)</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="p">):</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="p">):</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="p">):</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="p">):</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="p">):</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="p">):</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="p">):</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">return</span> <span class="n">arg</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">):</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">return</span> <span class="n">arg</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">return</span> <span class="n">arg</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a><span class="c1"># presto has a boolean type whereas tsql doesn't (people use bits)</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">for</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="n">arg</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="c1"># presto has a boolean type whereas tsql doesn't (people use bits)</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="k">for</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -342,16 +350,18 @@ conversions rely on type inference.</p>
|
|||
</span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
|
||||
</span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span>
|
||||
</span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a> <span class="p">):</span>
|
||||
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
||||
</span><span id="replace_date_funcs-47"><a href="#replace_date_funcs-47"><span class="linenos">47</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||
</span><span id="replace_date_funcs-48"><a href="#replace_date_funcs-48"><span class="linenos">48</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
|
||||
</span><span id="replace_date_funcs-49"><a href="#replace_date_funcs-49"><span class="linenos">49</span></a>
|
||||
</span><span id="replace_date_funcs-50"><a href="#replace_date_funcs-50"><span class="linenos">50</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-51"><a href="#replace_date_funcs-51"><span class="linenos">51</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-52"><a href="#replace_date_funcs-52"><span class="linenos">52</span></a>
|
||||
</span><span id="replace_date_funcs-53"><a href="#replace_date_funcs-53"><span class="linenos">53</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">is_string</span>
|
||||
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="ow">and</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="p">):</span>
|
||||
</span><span id="replace_date_funcs-47"><a href="#replace_date_funcs-47"><span class="linenos">47</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-48"><a href="#replace_date_funcs-48"><span class="linenos">48</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
||||
</span><span id="replace_date_funcs-49"><a href="#replace_date_funcs-49"><span class="linenos">49</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||
</span><span id="replace_date_funcs-50"><a href="#replace_date_funcs-50"><span class="linenos">50</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
|
||||
</span><span id="replace_date_funcs-51"><a href="#replace_date_funcs-51"><span class="linenos">51</span></a>
|
||||
</span><span id="replace_date_funcs-52"><a href="#replace_date_funcs-52"><span class="linenos">52</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-53"><a href="#replace_date_funcs-53"><span class="linenos">53</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
||||
</span><span id="replace_date_funcs-54"><a href="#replace_date_funcs-54"><span class="linenos">54</span></a>
|
||||
</span><span id="replace_date_funcs-55"><a href="#replace_date_funcs-55"><span class="linenos">55</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -382,21 +392,21 @@ conversions rely on type inference.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#coerce_type"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
||||
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
||||
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
||||
</span><span id="coerce_type-75"><a href="#coerce_type-75"><span class="linenos">75</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
||||
</span><span id="coerce_type-76"><a href="#coerce_type-76"><span class="linenos">76</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="coerce_type-77"><a href="#coerce_type-77"><span class="linenos">77</span></a> <span class="p">):</span>
|
||||
</span><span id="coerce_type-78"><a href="#coerce_type-78"><span class="linenos">78</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-79"><a href="#coerce_type-79"><span class="linenos">79</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
||||
</span><span id="coerce_type-80"><a href="#coerce_type-80"><span class="linenos">80</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-81"><a href="#coerce_type-81"><span class="linenos">81</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
||||
</span><span id="coerce_type-82"><a href="#coerce_type-82"><span class="linenos">82</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-83"><a href="#coerce_type-83"><span class="linenos">83</span></a>
|
||||
</span><span id="coerce_type-84"><a href="#coerce_type-84"><span class="linenos">84</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
||||
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-75"><a href="#coerce_type-75"><span class="linenos">75</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
||||
</span><span id="coerce_type-76"><a href="#coerce_type-76"><span class="linenos">76</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
||||
</span><span id="coerce_type-77"><a href="#coerce_type-77"><span class="linenos">77</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
||||
</span><span id="coerce_type-78"><a href="#coerce_type-78"><span class="linenos">78</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="coerce_type-79"><a href="#coerce_type-79"><span class="linenos">79</span></a> <span class="p">):</span>
|
||||
</span><span id="coerce_type-80"><a href="#coerce_type-80"><span class="linenos">80</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-81"><a href="#coerce_type-81"><span class="linenos">81</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
||||
</span><span id="coerce_type-82"><a href="#coerce_type-82"><span class="linenos">82</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-83"><a href="#coerce_type-83"><span class="linenos">83</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
||||
</span><span id="coerce_type-84"><a href="#coerce_type-84"><span class="linenos">84</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="coerce_type-85"><a href="#coerce_type-85"><span class="linenos">85</span></a>
|
||||
</span><span id="coerce_type-86"><a href="#coerce_type-86"><span class="linenos">86</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -414,14 +424,20 @@ conversions rely on type inference.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#remove_redundant_casts"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-87"><a href="#remove_redundant_casts-87"><span class="linenos">87</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_redundant_casts-88"><a href="#remove_redundant_casts-88"><span class="linenos">88</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="remove_redundant_casts-89"><a href="#remove_redundant_casts-89"><span class="linenos">89</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
||||
</span><span id="remove_redundant_casts-90"><a href="#remove_redundant_casts-90"><span class="linenos">90</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="remove_redundant_casts-91"><a href="#remove_redundant_casts-91"><span class="linenos">91</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_redundant_casts-92"><a href="#remove_redundant_casts-92"><span class="linenos">92</span></a> <span class="p">):</span>
|
||||
</span><span id="remove_redundant_casts-93"><a href="#remove_redundant_casts-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_redundant_casts-94"><a href="#remove_redundant_casts-94"><span class="linenos">94</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-89"><a href="#remove_redundant_casts-89"><span class="linenos"> 89</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_redundant_casts-90"><a href="#remove_redundant_casts-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="remove_redundant_casts-91"><a href="#remove_redundant_casts-91"><span class="linenos"> 91</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
||||
</span><span id="remove_redundant_casts-92"><a href="#remove_redundant_casts-92"><span class="linenos"> 92</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="remove_redundant_casts-93"><a href="#remove_redundant_casts-93"><span class="linenos"> 93</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_redundant_casts-94"><a href="#remove_redundant_casts-94"><span class="linenos"> 94</span></a> <span class="p">):</span>
|
||||
</span><span id="remove_redundant_casts-95"><a href="#remove_redundant_casts-95"><span class="linenos"> 95</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_redundant_casts-96"><a href="#remove_redundant_casts-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="remove_redundant_casts-97"><a href="#remove_redundant_casts-97"><span class="linenos"> 97</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
|
||||
</span><span id="remove_redundant_casts-98"><a href="#remove_redundant_casts-98"><span class="linenos"> 98</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||
</span><span id="remove_redundant_casts-99"><a href="#remove_redundant_casts-99"><span class="linenos"> 99</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
|
||||
</span><span id="remove_redundant_casts-100"><a href="#remove_redundant_casts-100"><span class="linenos">100</span></a> <span class="p">):</span>
|
||||
</span><span id="remove_redundant_casts-101"><a href="#remove_redundant_casts-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_redundant_casts-102"><a href="#remove_redundant_casts-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -439,23 +455,23 @@ conversions rely on type inference.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#ensure_bools"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
||||
</span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
||||
</span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
||||
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-105"><a href="#ensure_bools-105"><span class="linenos">105</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
||||
</span><span id="ensure_bools-106"><a href="#ensure_bools-106"><span class="linenos">106</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="ensure_bools-107"><a href="#ensure_bools-107"><span class="linenos">107</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="ensure_bools-108"><a href="#ensure_bools-108"><span class="linenos">108</span></a> <span class="p">):</span>
|
||||
</span><span id="ensure_bools-109"><a href="#ensure_bools-109"><span class="linenos">109</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-110"><a href="#ensure_bools-110"><span class="linenos">110</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
||||
</span><span id="ensure_bools-111"><a href="#ensure_bools-111"><span class="linenos">111</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-112"><a href="#ensure_bools-112"><span class="linenos">112</span></a>
|
||||
</span><span id="ensure_bools-113"><a href="#ensure_bools-113"><span class="linenos">113</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-105"><a href="#ensure_bools-105"><span class="linenos">105</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
||||
</span><span id="ensure_bools-106"><a href="#ensure_bools-106"><span class="linenos">106</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
||||
</span><span id="ensure_bools-107"><a href="#ensure_bools-107"><span class="linenos">107</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="ensure_bools-108"><a href="#ensure_bools-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="ensure_bools-109"><a href="#ensure_bools-109"><span class="linenos">109</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-110"><a href="#ensure_bools-110"><span class="linenos">110</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-111"><a href="#ensure_bools-111"><span class="linenos">111</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
||||
</span><span id="ensure_bools-112"><a href="#ensure_bools-112"><span class="linenos">112</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-113"><a href="#ensure_bools-113"><span class="linenos">113</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
||||
</span><span id="ensure_bools-114"><a href="#ensure_bools-114"><span class="linenos">114</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="ensure_bools-115"><a href="#ensure_bools-115"><span class="linenos">115</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="ensure_bools-116"><a href="#ensure_bools-116"><span class="linenos">116</span></a> <span class="p">):</span>
|
||||
</span><span id="ensure_bools-117"><a href="#ensure_bools-117"><span class="linenos">117</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-118"><a href="#ensure_bools-118"><span class="linenos">118</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
||||
</span><span id="ensure_bools-119"><a href="#ensure_bools-119"><span class="linenos">119</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="ensure_bools-120"><a href="#ensure_bools-120"><span class="linenos">120</span></a>
|
||||
</span><span id="ensure_bools-121"><a href="#ensure_bools-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -473,12 +489,12 @@ conversions rely on type inference.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#remove_ascending_order"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-116"><a href="#remove_ascending_order-116"><span class="linenos">116</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_ascending_order-117"><a href="#remove_ascending_order-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
||||
</span><span id="remove_ascending_order-118"><a href="#remove_ascending_order-118"><span class="linenos">118</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
||||
</span><span id="remove_ascending_order-119"><a href="#remove_ascending_order-119"><span class="linenos">119</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="remove_ascending_order-120"><a href="#remove_ascending_order-120"><span class="linenos">120</span></a>
|
||||
</span><span id="remove_ascending_order-121"><a href="#remove_ascending_order-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-124"><a href="#remove_ascending_order-124"><span class="linenos">124</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_ascending_order-125"><a href="#remove_ascending_order-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
||||
</span><span id="remove_ascending_order-126"><a href="#remove_ascending_order-126"><span class="linenos">126</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
||||
</span><span id="remove_ascending_order-127"><a href="#remove_ascending_order-127"><span class="linenos">127</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="remove_ascending_order-128"><a href="#remove_ascending_order-128"><span class="linenos">128</span></a>
|
||||
</span><span id="remove_ascending_order-129"><a href="#remove_ascending_order-129"><span class="linenos">129</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
@ -586,7 +586,7 @@ queries if it would result in multiple table selects in a single query:</p>
|
|||
<div class="attr variable">
|
||||
<span class="name">UNMERGABLE_ARGS</span> =
|
||||
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'limit', 'with', 'into', 'match', 'pivots', 'group', 'locks', 'prewhere', 'kind', 'connect', 'having', 'settings', 'windows', 'options', 'sort', 'laterals', 'qualify', 'format', 'cluster', 'distinct', 'offset', 'sample', 'distribute'}</span>
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'connect', 'options', 'group', 'having', 'qualify', 'sample', 'offset', 'with', 'cluster', 'limit', 'settings', 'distinct', 'sort', 'match', 'windows', 'format', 'pivots', 'kind', 'prewhere', 'into', 'laterals', 'locks', 'distribute'}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
|
@ -77,45 +77,49 @@
|
|||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a><span class="sd"> on the dialect. This essentially makes those identifiers case-insensitive.</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a><span class="sd"> It's possible to make this a no-op by adding a special comment next to the</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a><span class="sd"> identifier of interest:</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> Normalize identifiers by converting them to either lower or upper case,</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a><span class="sd"> ensuring the semantics are preserved in each case (e.g. by respecting</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a><span class="sd"> case-sensitivity).</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a><span class="sd"> This transformation reflects how identifiers would be resolved by the engine corresponding</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a><span class="sd"> to each SQL dialect, and plays a very important role in the standardization of the AST.</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a><span class="sd"> It's possible to make this a no-op by adding a special comment next to the</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a><span class="sd"> identifier of interest:</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a><span class="sd"> Note:</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a><span class="sd"> Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="sd"> when they're quoted, so in these cases all identifiers are normalized.</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> >>> normalize_identifiers("foo", dialect="snowflake").sql(dialect="snowflake")</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> 'FOO'</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a><span class="sd"> Note:</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a><span class="sd"> when they're quoted, so in these cases all identifiers are normalized.</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="sd"> >>> normalize_identifiers("foo", dialect="snowflake").sql(dialect="snowflake")</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> 'FOO'</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">)):</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">):</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">)):</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">):</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -133,50 +137,58 @@
|
|||
<a class="headerlink" href="#normalize_identifiers"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize_identifiers-21"><a href="#normalize_identifiers-21"><span class="linenos">21</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="normalize_identifiers-22"><a href="#normalize_identifiers-22"><span class="linenos">22</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="normalize_identifiers-23"><a href="#normalize_identifiers-23"><span class="linenos">23</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending</span>
|
||||
</span><span id="normalize_identifiers-24"><a href="#normalize_identifiers-24"><span class="linenos">24</span></a><span class="sd"> on the dialect. This essentially makes those identifiers case-insensitive.</span>
|
||||
</span><span id="normalize_identifiers-25"><a href="#normalize_identifiers-25"><span class="linenos">25</span></a>
|
||||
</span><span id="normalize_identifiers-26"><a href="#normalize_identifiers-26"><span class="linenos">26</span></a><span class="sd"> It's possible to make this a no-op by adding a special comment next to the</span>
|
||||
</span><span id="normalize_identifiers-27"><a href="#normalize_identifiers-27"><span class="linenos">27</span></a><span class="sd"> identifier of interest:</span>
|
||||
</span><span id="normalize_identifiers-28"><a href="#normalize_identifiers-28"><span class="linenos">28</span></a>
|
||||
</span><span id="normalize_identifiers-29"><a href="#normalize_identifiers-29"><span class="linenos">29</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
|
||||
</span><span id="normalize_identifiers-30"><a href="#normalize_identifiers-30"><span class="linenos">30</span></a>
|
||||
</span><span id="normalize_identifiers-31"><a href="#normalize_identifiers-31"><span class="linenos">31</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
|
||||
</span><span id="normalize_identifiers-23"><a href="#normalize_identifiers-23"><span class="linenos">23</span></a><span class="sd"> Normalize identifiers by converting them to either lower or upper case,</span>
|
||||
</span><span id="normalize_identifiers-24"><a href="#normalize_identifiers-24"><span class="linenos">24</span></a><span class="sd"> ensuring the semantics are preserved in each case (e.g. by respecting</span>
|
||||
</span><span id="normalize_identifiers-25"><a href="#normalize_identifiers-25"><span class="linenos">25</span></a><span class="sd"> case-sensitivity).</span>
|
||||
</span><span id="normalize_identifiers-26"><a href="#normalize_identifiers-26"><span class="linenos">26</span></a>
|
||||
</span><span id="normalize_identifiers-27"><a href="#normalize_identifiers-27"><span class="linenos">27</span></a><span class="sd"> This transformation reflects how identifiers would be resolved by the engine corresponding</span>
|
||||
</span><span id="normalize_identifiers-28"><a href="#normalize_identifiers-28"><span class="linenos">28</span></a><span class="sd"> to each SQL dialect, and plays a very important role in the standardization of the AST.</span>
|
||||
</span><span id="normalize_identifiers-29"><a href="#normalize_identifiers-29"><span class="linenos">29</span></a>
|
||||
</span><span id="normalize_identifiers-30"><a href="#normalize_identifiers-30"><span class="linenos">30</span></a><span class="sd"> It's possible to make this a no-op by adding a special comment next to the</span>
|
||||
</span><span id="normalize_identifiers-31"><a href="#normalize_identifiers-31"><span class="linenos">31</span></a><span class="sd"> identifier of interest:</span>
|
||||
</span><span id="normalize_identifiers-32"><a href="#normalize_identifiers-32"><span class="linenos">32</span></a>
|
||||
</span><span id="normalize_identifiers-33"><a href="#normalize_identifiers-33"><span class="linenos">33</span></a><span class="sd"> Note:</span>
|
||||
</span><span id="normalize_identifiers-34"><a href="#normalize_identifiers-34"><span class="linenos">34</span></a><span class="sd"> Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even</span>
|
||||
</span><span id="normalize_identifiers-35"><a href="#normalize_identifiers-35"><span class="linenos">35</span></a><span class="sd"> when they're quoted, so in these cases all identifiers are normalized.</span>
|
||||
</span><span id="normalize_identifiers-33"><a href="#normalize_identifiers-33"><span class="linenos">33</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
|
||||
</span><span id="normalize_identifiers-34"><a href="#normalize_identifiers-34"><span class="linenos">34</span></a>
|
||||
</span><span id="normalize_identifiers-35"><a href="#normalize_identifiers-35"><span class="linenos">35</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
|
||||
</span><span id="normalize_identifiers-36"><a href="#normalize_identifiers-36"><span class="linenos">36</span></a>
|
||||
</span><span id="normalize_identifiers-37"><a href="#normalize_identifiers-37"><span class="linenos">37</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="normalize_identifiers-38"><a href="#normalize_identifiers-38"><span class="linenos">38</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="normalize_identifiers-39"><a href="#normalize_identifiers-39"><span class="linenos">39</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="normalize_identifiers-40"><a href="#normalize_identifiers-40"><span class="linenos">40</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="normalize_identifiers-41"><a href="#normalize_identifiers-41"><span class="linenos">41</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="normalize_identifiers-42"><a href="#normalize_identifiers-42"><span class="linenos">42</span></a><span class="sd"> >>> normalize_identifiers("foo", dialect="snowflake").sql(dialect="snowflake")</span>
|
||||
</span><span id="normalize_identifiers-43"><a href="#normalize_identifiers-43"><span class="linenos">43</span></a><span class="sd"> 'FOO'</span>
|
||||
</span><span id="normalize_identifiers-44"><a href="#normalize_identifiers-44"><span class="linenos">44</span></a>
|
||||
</span><span id="normalize_identifiers-45"><a href="#normalize_identifiers-45"><span class="linenos">45</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="normalize_identifiers-46"><a href="#normalize_identifiers-46"><span class="linenos">46</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="normalize_identifiers-47"><a href="#normalize_identifiers-47"><span class="linenos">47</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="normalize_identifiers-37"><a href="#normalize_identifiers-37"><span class="linenos">37</span></a><span class="sd"> Note:</span>
|
||||
</span><span id="normalize_identifiers-38"><a href="#normalize_identifiers-38"><span class="linenos">38</span></a><span class="sd"> Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even</span>
|
||||
</span><span id="normalize_identifiers-39"><a href="#normalize_identifiers-39"><span class="linenos">39</span></a><span class="sd"> when they're quoted, so in these cases all identifiers are normalized.</span>
|
||||
</span><span id="normalize_identifiers-40"><a href="#normalize_identifiers-40"><span class="linenos">40</span></a>
|
||||
</span><span id="normalize_identifiers-41"><a href="#normalize_identifiers-41"><span class="linenos">41</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="normalize_identifiers-42"><a href="#normalize_identifiers-42"><span class="linenos">42</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="normalize_identifiers-43"><a href="#normalize_identifiers-43"><span class="linenos">43</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="normalize_identifiers-44"><a href="#normalize_identifiers-44"><span class="linenos">44</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="normalize_identifiers-45"><a href="#normalize_identifiers-45"><span class="linenos">45</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="normalize_identifiers-46"><a href="#normalize_identifiers-46"><span class="linenos">46</span></a><span class="sd"> >>> normalize_identifiers("foo", dialect="snowflake").sql(dialect="snowflake")</span>
|
||||
</span><span id="normalize_identifiers-47"><a href="#normalize_identifiers-47"><span class="linenos">47</span></a><span class="sd"> 'FOO'</span>
|
||||
</span><span id="normalize_identifiers-48"><a href="#normalize_identifiers-48"><span class="linenos">48</span></a>
|
||||
</span><span id="normalize_identifiers-49"><a href="#normalize_identifiers-49"><span class="linenos">49</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> """</span>
|
||||
</span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a>
|
||||
</span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a>
|
||||
</span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">)):</span>
|
||||
</span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">):</span>
|
||||
</span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="normalize_identifiers-49"><a href="#normalize_identifiers-49"><span class="linenos">49</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a>
|
||||
</span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a><span class="sd"> """</span>
|
||||
</span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a>
|
||||
</span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="normalize_identifiers-60"><a href="#normalize_identifiers-60"><span class="linenos">60</span></a>
|
||||
</span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">)):</span>
|
||||
</span><span id="normalize_identifiers-62"><a href="#normalize_identifiers-62"><span class="linenos">62</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"case_sensitive"</span><span class="p">):</span>
|
||||
</span><span id="normalize_identifiers-63"><a href="#normalize_identifiers-63"><span class="linenos">63</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="normalize_identifiers-64"><a href="#normalize_identifiers-64"><span class="linenos">64</span></a>
|
||||
</span><span id="normalize_identifiers-65"><a href="#normalize_identifiers-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Normalize all unquoted identifiers to either lower or upper case, depending
|
||||
on the dialect. This essentially makes those identifiers case-insensitive.</p>
|
||||
<div class="docstring"><p>Normalize identifiers by converting them to either lower or upper case,
|
||||
ensuring the semantics are preserved in each case (e.g. by respecting
|
||||
case-sensitivity).</p>
|
||||
|
||||
<p>This transformation reflects how identifiers would be resolved by the engine corresponding
|
||||
to each SQL dialect, and plays a very important role in the standardization of the AST.</p>
|
||||
|
||||
<p>It's possible to make this a no-op by adding a special comment next to the
|
||||
identifier of interest:</p>
|
||||
|
@ -189,7 +201,7 @@ identifier of interest:</p>
|
|||
<h6 id="note">Note:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even
|
||||
<p>Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even
|
||||
when they're quoted, so in these cases all identifiers are normalized.</p>
|
||||
</blockquote>
|
||||
|
||||
|
|
|
@ -65,89 +65,98 @@
|
|||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.isolate_table_selects</span> <span class="kn">import</span> <span class="n">isolate_table_selects</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.normalize_identifiers</span> <span class="kn">import</span> <span class="n">normalize_identifiers</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.qualify_columns</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a> <span class="n">pushdown_cte_alias_columns</span> <span class="k">as</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">,</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a> <span class="n">qualify_columns</span> <span class="k">as</span> <span class="n">qualify_columns_func</span><span class="p">,</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a> <span class="n">quote_identifiers</span> <span class="k">as</span> <span class="n">quote_identifiers_func</span><span class="p">,</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a> <span class="n">validate_qualify_columns</span> <span class="k">as</span> <span class="n">validate_qualify_columns_func</span><span class="p">,</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a><span class="p">)</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.qualify_tables</span> <span class="kn">import</span> <span class="n">qualify_tables</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span><span class="p">,</span> <span class="n">ensure_schema</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> >>> schema = {"tbl": {"col": "INT"}}</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT col FROM tbl")</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a><span class="sd"> >>> qualify(expression, schema=schema).sql()</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a><span class="sd"> expression: Expression to qualify.</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> db: Default database name for tables.</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="sd"> for most of the optimizer's rules to work; do not set to False unless you</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a><span class="sd"> know what you're doing!</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">schema</span><span class="p">,</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a> <span class="n">pushdown_cte_alias_columns</span> <span class="k">as</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">,</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> <span class="n">qualify_columns</span> <span class="k">as</span> <span class="n">qualify_columns_func</span><span class="p">,</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a> <span class="n">quote_identifiers</span> <span class="k">as</span> <span class="n">quote_identifiers_func</span><span class="p">,</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="n">validate_qualify_columns</span> <span class="k">as</span> <span class="n">validate_qualify_columns_func</span><span class="p">,</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="p">)</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.qualify_tables</span> <span class="kn">import</span> <span class="n">qualify_tables</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span><span class="p">,</span> <span class="n">ensure_schema</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> >>> schema = {"tbl": {"col": "INT"}}</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT col FROM tbl")</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> >>> qualify(expression, schema=schema).sql()</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="sd"> 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="sd"> expression: Expression to qualify.</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a><span class="sd"> db: Default database name for tables.</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a><span class="sd"> for most of the optimizer's rules to work; do not set to False unless you</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="sd"> know what you're doing!</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="n">infer_csv_schemas</span><span class="p">,</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">schema</span><span class="p">,</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="p">)</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -157,86 +166,95 @@
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">qualify</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
|
||||
<span class="name">qualify</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="qualify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#qualify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify-20"><a href="#qualify-20"><span class="linenos">20</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
|
||||
</span><span id="qualify-21"><a href="#qualify-21"><span class="linenos">21</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="qualify-22"><a href="#qualify-22"><span class="linenos">22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-23"><a href="#qualify-23"><span class="linenos">23</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-24"><a href="#qualify-24"><span class="linenos">24</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-25"><a href="#qualify-25"><span class="linenos">25</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-26"><a href="#qualify-26"><span class="linenos">26</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-27"><a href="#qualify-27"><span class="linenos">27</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-28"><a href="#qualify-28"><span class="linenos">28</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-29"><a href="#qualify-29"><span class="linenos">29</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="qualify-30"><a href="#qualify-30"><span class="linenos">30</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-31"><a href="#qualify-31"><span class="linenos">31</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-32"><a href="#qualify-32"><span class="linenos">32</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-33"><a href="#qualify-33"><span class="linenos">33</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-34"><a href="#qualify-34"><span class="linenos">34</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="qualify-35"><a href="#qualify-35"><span class="linenos">35</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="qualify-36"><a href="#qualify-36"><span class="linenos">36</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
|
||||
</span><span id="qualify-37"><a href="#qualify-37"><span class="linenos">37</span></a>
|
||||
</span><span id="qualify-38"><a href="#qualify-38"><span class="linenos">38</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
|
||||
</span><span id="qualify-39"><a href="#qualify-39"><span class="linenos">39</span></a>
|
||||
</span><span id="qualify-40"><a href="#qualify-40"><span class="linenos">40</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="qualify-41"><a href="#qualify-41"><span class="linenos">41</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="qualify-42"><a href="#qualify-42"><span class="linenos">42</span></a><span class="sd"> >>> schema = {"tbl": {"col": "INT"}}</span>
|
||||
</span><span id="qualify-43"><a href="#qualify-43"><span class="linenos">43</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT col FROM tbl")</span>
|
||||
</span><span id="qualify-44"><a href="#qualify-44"><span class="linenos">44</span></a><span class="sd"> >>> qualify(expression, schema=schema).sql()</span>
|
||||
</span><span id="qualify-45"><a href="#qualify-45"><span class="linenos">45</span></a><span class="sd"> 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'</span>
|
||||
</span><span id="qualify-46"><a href="#qualify-46"><span class="linenos">46</span></a>
|
||||
</span><span id="qualify-47"><a href="#qualify-47"><span class="linenos">47</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="qualify-48"><a href="#qualify-48"><span class="linenos">48</span></a><span class="sd"> expression: Expression to qualify.</span>
|
||||
</span><span id="qualify-49"><a href="#qualify-49"><span class="linenos">49</span></a><span class="sd"> db: Default database name for tables.</span>
|
||||
</span><span id="qualify-50"><a href="#qualify-50"><span class="linenos">50</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
|
||||
</span><span id="qualify-51"><a href="#qualify-51"><span class="linenos">51</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
|
||||
</span><span id="qualify-52"><a href="#qualify-52"><span class="linenos">52</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
|
||||
</span><span id="qualify-53"><a href="#qualify-53"><span class="linenos">53</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
|
||||
</span><span id="qualify-54"><a href="#qualify-54"><span class="linenos">54</span></a><span class="sd"> for most of the optimizer's rules to work; do not set to False unless you</span>
|
||||
</span><span id="qualify-55"><a href="#qualify-55"><span class="linenos">55</span></a><span class="sd"> know what you're doing!</span>
|
||||
</span><span id="qualify-56"><a href="#qualify-56"><span class="linenos">56</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
|
||||
</span><span id="qualify-57"><a href="#qualify-57"><span class="linenos">57</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
|
||||
</span><span id="qualify-58"><a href="#qualify-58"><span class="linenos">58</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
|
||||
</span><span id="qualify-59"><a href="#qualify-59"><span class="linenos">59</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
|
||||
</span><span id="qualify-60"><a href="#qualify-60"><span class="linenos">60</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
|
||||
</span><span id="qualify-61"><a href="#qualify-61"><span class="linenos">61</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
|
||||
</span><span id="qualify-62"><a href="#qualify-62"><span class="linenos">62</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
|
||||
</span><span id="qualify-63"><a href="#qualify-63"><span class="linenos">63</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
|
||||
</span><span id="qualify-64"><a href="#qualify-64"><span class="linenos">64</span></a>
|
||||
</span><span id="qualify-65"><a href="#qualify-65"><span class="linenos">65</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="qualify-66"><a href="#qualify-66"><span class="linenos">66</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="qualify-67"><a href="#qualify-67"><span class="linenos">67</span></a><span class="sd"> """</span>
|
||||
</span><span id="qualify-68"><a href="#qualify-68"><span class="linenos">68</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="qualify-69"><a href="#qualify-69"><span class="linenos">69</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="qualify-70"><a href="#qualify-70"><span class="linenos">70</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="qualify-71"><a href="#qualify-71"><span class="linenos">71</span></a>
|
||||
</span><span id="qualify-72"><a href="#qualify-72"><span class="linenos">72</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
|
||||
</span><span id="qualify-73"><a href="#qualify-73"><span class="linenos">73</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="qualify-74"><a href="#qualify-74"><span class="linenos">74</span></a>
|
||||
</span><span id="qualify-75"><a href="#qualify-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
|
||||
</span><span id="qualify-76"><a href="#qualify-76"><span class="linenos">76</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="qualify-77"><a href="#qualify-77"><span class="linenos">77</span></a>
|
||||
</span><span id="qualify-78"><a href="#qualify-78"><span class="linenos">78</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="qualify-79"><a href="#qualify-79"><span class="linenos">79</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
|
||||
</span><span id="qualify-80"><a href="#qualify-80"><span class="linenos">80</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="qualify-81"><a href="#qualify-81"><span class="linenos">81</span></a> <span class="n">schema</span><span class="p">,</span>
|
||||
</span><span id="qualify-82"><a href="#qualify-82"><span class="linenos">82</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
|
||||
</span><span id="qualify-83"><a href="#qualify-83"><span class="linenos">83</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
|
||||
</span><span id="qualify-84"><a href="#qualify-84"><span class="linenos">84</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
|
||||
</span><span id="qualify-85"><a href="#qualify-85"><span class="linenos">85</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify-86"><a href="#qualify-86"><span class="linenos">86</span></a>
|
||||
</span><span id="qualify-87"><a href="#qualify-87"><span class="linenos">87</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
|
||||
</span><span id="qualify-88"><a href="#qualify-88"><span class="linenos">88</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
|
||||
</span><span id="qualify-89"><a href="#qualify-89"><span class="linenos">89</span></a>
|
||||
</span><span id="qualify-90"><a href="#qualify-90"><span class="linenos">90</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="qualify-91"><a href="#qualify-91"><span class="linenos">91</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="qualify-92"><a href="#qualify-92"><span class="linenos">92</span></a>
|
||||
</span><span id="qualify-93"><a href="#qualify-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify-20"><a href="#qualify-20"><span class="linenos"> 20</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
|
||||
</span><span id="qualify-21"><a href="#qualify-21"><span class="linenos"> 21</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="qualify-22"><a href="#qualify-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-23"><a href="#qualify-23"><span class="linenos"> 23</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-24"><a href="#qualify-24"><span class="linenos"> 24</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-25"><a href="#qualify-25"><span class="linenos"> 25</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-26"><a href="#qualify-26"><span class="linenos"> 26</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-27"><a href="#qualify-27"><span class="linenos"> 27</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-28"><a href="#qualify-28"><span class="linenos"> 28</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify-29"><a href="#qualify-29"><span class="linenos"> 29</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="qualify-30"><a href="#qualify-30"><span class="linenos"> 30</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-31"><a href="#qualify-31"><span class="linenos"> 31</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-32"><a href="#qualify-32"><span class="linenos"> 32</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-33"><a href="#qualify-33"><span class="linenos"> 33</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify-34"><a href="#qualify-34"><span class="linenos"> 34</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="qualify-35"><a href="#qualify-35"><span class="linenos"> 35</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="qualify-36"><a href="#qualify-36"><span class="linenos"> 36</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="qualify-37"><a href="#qualify-37"><span class="linenos"> 37</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
|
||||
</span><span id="qualify-38"><a href="#qualify-38"><span class="linenos"> 38</span></a>
|
||||
</span><span id="qualify-39"><a href="#qualify-39"><span class="linenos"> 39</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
|
||||
</span><span id="qualify-40"><a href="#qualify-40"><span class="linenos"> 40</span></a>
|
||||
</span><span id="qualify-41"><a href="#qualify-41"><span class="linenos"> 41</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="qualify-42"><a href="#qualify-42"><span class="linenos"> 42</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="qualify-43"><a href="#qualify-43"><span class="linenos"> 43</span></a><span class="sd"> >>> schema = {"tbl": {"col": "INT"}}</span>
|
||||
</span><span id="qualify-44"><a href="#qualify-44"><span class="linenos"> 44</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT col FROM tbl")</span>
|
||||
</span><span id="qualify-45"><a href="#qualify-45"><span class="linenos"> 45</span></a><span class="sd"> >>> qualify(expression, schema=schema).sql()</span>
|
||||
</span><span id="qualify-46"><a href="#qualify-46"><span class="linenos"> 46</span></a><span class="sd"> 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'</span>
|
||||
</span><span id="qualify-47"><a href="#qualify-47"><span class="linenos"> 47</span></a>
|
||||
</span><span id="qualify-48"><a href="#qualify-48"><span class="linenos"> 48</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="qualify-49"><a href="#qualify-49"><span class="linenos"> 49</span></a><span class="sd"> expression: Expression to qualify.</span>
|
||||
</span><span id="qualify-50"><a href="#qualify-50"><span class="linenos"> 50</span></a><span class="sd"> db: Default database name for tables.</span>
|
||||
</span><span id="qualify-51"><a href="#qualify-51"><span class="linenos"> 51</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
|
||||
</span><span id="qualify-52"><a href="#qualify-52"><span class="linenos"> 52</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
|
||||
</span><span id="qualify-53"><a href="#qualify-53"><span class="linenos"> 53</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
|
||||
</span><span id="qualify-54"><a href="#qualify-54"><span class="linenos"> 54</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
|
||||
</span><span id="qualify-55"><a href="#qualify-55"><span class="linenos"> 55</span></a><span class="sd"> for most of the optimizer's rules to work; do not set to False unless you</span>
|
||||
</span><span id="qualify-56"><a href="#qualify-56"><span class="linenos"> 56</span></a><span class="sd"> know what you're doing!</span>
|
||||
</span><span id="qualify-57"><a href="#qualify-57"><span class="linenos"> 57</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
|
||||
</span><span id="qualify-58"><a href="#qualify-58"><span class="linenos"> 58</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
|
||||
</span><span id="qualify-59"><a href="#qualify-59"><span class="linenos"> 59</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
|
||||
</span><span id="qualify-60"><a href="#qualify-60"><span class="linenos"> 60</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
|
||||
</span><span id="qualify-61"><a href="#qualify-61"><span class="linenos"> 61</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
|
||||
</span><span id="qualify-62"><a href="#qualify-62"><span class="linenos"> 62</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
|
||||
</span><span id="qualify-63"><a href="#qualify-63"><span class="linenos"> 63</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
|
||||
</span><span id="qualify-64"><a href="#qualify-64"><span class="linenos"> 64</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
|
||||
</span><span id="qualify-65"><a href="#qualify-65"><span class="linenos"> 65</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</span>
|
||||
</span><span id="qualify-66"><a href="#qualify-66"><span class="linenos"> 66</span></a>
|
||||
</span><span id="qualify-67"><a href="#qualify-67"><span class="linenos"> 67</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="qualify-68"><a href="#qualify-68"><span class="linenos"> 68</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="qualify-69"><a href="#qualify-69"><span class="linenos"> 69</span></a><span class="sd"> """</span>
|
||||
</span><span id="qualify-70"><a href="#qualify-70"><span class="linenos"> 70</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="qualify-71"><a href="#qualify-71"><span class="linenos"> 71</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="qualify-72"><a href="#qualify-72"><span class="linenos"> 72</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span>
|
||||
</span><span id="qualify-73"><a href="#qualify-73"><span class="linenos"> 73</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="qualify-74"><a href="#qualify-74"><span class="linenos"> 74</span></a> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span>
|
||||
</span><span id="qualify-75"><a href="#qualify-75"><span class="linenos"> 75</span></a> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span>
|
||||
</span><span id="qualify-76"><a href="#qualify-76"><span class="linenos"> 76</span></a> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span>
|
||||
</span><span id="qualify-77"><a href="#qualify-77"><span class="linenos"> 77</span></a> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span>
|
||||
</span><span id="qualify-78"><a href="#qualify-78"><span class="linenos"> 78</span></a> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="n">infer_csv_schemas</span><span class="p">,</span>
|
||||
</span><span id="qualify-79"><a href="#qualify-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify-80"><a href="#qualify-80"><span class="linenos"> 80</span></a>
|
||||
</span><span id="qualify-81"><a href="#qualify-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
|
||||
</span><span id="qualify-82"><a href="#qualify-82"><span class="linenos"> 82</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="qualify-83"><a href="#qualify-83"><span class="linenos"> 83</span></a>
|
||||
</span><span id="qualify-84"><a href="#qualify-84"><span class="linenos"> 84</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
|
||||
</span><span id="qualify-85"><a href="#qualify-85"><span class="linenos"> 85</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="qualify-86"><a href="#qualify-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="qualify-87"><a href="#qualify-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="qualify-88"><a href="#qualify-88"><span class="linenos"> 88</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
|
||||
</span><span id="qualify-89"><a href="#qualify-89"><span class="linenos"> 89</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="qualify-90"><a href="#qualify-90"><span class="linenos"> 90</span></a> <span class="n">schema</span><span class="p">,</span>
|
||||
</span><span id="qualify-91"><a href="#qualify-91"><span class="linenos"> 91</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
|
||||
</span><span id="qualify-92"><a href="#qualify-92"><span class="linenos"> 92</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
|
||||
</span><span id="qualify-93"><a href="#qualify-93"><span class="linenos"> 93</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
|
||||
</span><span id="qualify-94"><a href="#qualify-94"><span class="linenos"> 94</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify-95"><a href="#qualify-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="qualify-96"><a href="#qualify-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
|
||||
</span><span id="qualify-97"><a href="#qualify-97"><span class="linenos"> 97</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
|
||||
</span><span id="qualify-98"><a href="#qualify-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="qualify-99"><a href="#qualify-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
|
||||
</span><span id="qualify-100"><a href="#qualify-100"><span class="linenos">100</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="qualify-101"><a href="#qualify-101"><span class="linenos">101</span></a>
|
||||
</span><span id="qualify-102"><a href="#qualify-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -276,6 +294,7 @@ know what you're doing!</li>
|
|||
This step is necessary to ensure correctness for case sensitive queries.
|
||||
But this flag is provided in case this step is performed at a later time.</li>
|
||||
<li><strong>identify:</strong> If True, quote all identifiers, else only necessary ones.</li>
|
||||
<li><strong>infer_csv_schemas:</strong> Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
|
File diff suppressed because it is too large
Load diff
|
@ -76,131 +76,133 @@
|
|||
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> >>></span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> 'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> >>></span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> 'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="c1"># Don't add the pivot's alias to the pivoted table, use the table's name instead</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="p">)</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="c1"># Don't add the pivot's alias to the pivoted table, use the table's name instead</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">)</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="p">)</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="p">)</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="p">)</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="p">):</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="p">)</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="p">)</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="p">)</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="p">):</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -210,7 +212,7 @@
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">qualify_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="o">~</span><span class="n">E</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="o">~</span><span class="n">E</span>:</span></span>
|
||||
<span class="name">qualify_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="o">~</span><span class="n">E</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="o">~</span><span class="n">E</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="qualify_tables-view-source"><span>View Source</span></label>
|
||||
|
||||
|
@ -221,131 +223,133 @@
|
|||
</span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
|
||||
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
|
||||
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a>
|
||||
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> >>></span>
|
||||
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")</span>
|
||||
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> 'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a>
|
||||
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
|
||||
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a>
|
||||
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a><span class="sd"> """</span>
|
||||
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a>
|
||||
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a>
|
||||
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
|
||||
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a>
|
||||
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a>
|
||||
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a>
|
||||
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a>
|
||||
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
|
||||
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
|
||||
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a>
|
||||
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> >>></span>
|
||||
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")</span>
|
||||
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a><span class="sd"> 'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a>
|
||||
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</span>
|
||||
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
|
||||
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a>
|
||||
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a><span class="sd"> """</span>
|
||||
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
|
||||
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a>
|
||||
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a>
|
||||
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
|
||||
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a>
|
||||
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a>
|
||||
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a>
|
||||
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a>
|
||||
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="c1"># Don't add the pivot's alias to the pivoted table, use the table's name instead</span>
|
||||
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a>
|
||||
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
|
||||
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="c1"># Don't add the pivot's alias to the pivoted table, use the table's name instead</span>
|
||||
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a>
|
||||
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
|
||||
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a>
|
||||
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a>
|
||||
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a>
|
||||
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a> <span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
|
||||
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a>
|
||||
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="p">):</span>
|
||||
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
|
||||
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a>
|
||||
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
|
||||
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a>
|
||||
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a>
|
||||
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a>
|
||||
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a>
|
||||
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
|
||||
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a>
|
||||
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="p">):</span>
|
||||
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
|
||||
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a>
|
||||
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"."</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
|
||||
</span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a>
|
||||
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-147"><a href="#qualify_tables-147"><span class="linenos">147</span></a>
|
||||
</span><span id="qualify_tables-148"><a href="#qualify_tables-148"><span class="linenos">148</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
@ -375,6 +379,7 @@
|
|||
<li><strong>db:</strong> Database name</li>
|
||||
<li><strong>catalog:</strong> Catalog name</li>
|
||||
<li><strong>schema:</strong> A schema to populate</li>
|
||||
<li><strong>infer_csv_schemas:</strong> Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</li>
|
||||
<li><strong>dialect:</strong> The dialect to parse catalog and schema into.</li>
|
||||
</ul>
|
||||
|
||||
|
|
File diff suppressed because it is too large
Load diff
|
@ -3209,7 +3209,7 @@ prefix are statically known.</p>
|
|||
<div class="attr variable">
|
||||
<span class="name">DATETRUNC_COMPARISONS</span> =
|
||||
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{<class '<a href="../expressions.html#GT">sqlglot.expressions.GT</a>'>, <class '<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>'>, <class '<a href="../expressions.html#LT">sqlglot.expressions.LT</a>'>, <class '<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>'>, <class '<a href="../expressions.html#In">sqlglot.expressions.In</a>'>, <class '<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>'>, <class '<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>'>}</span>
|
||||
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{<class '<a href="../expressions.html#LT">sqlglot.expressions.LT</a>'>, <class '<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>'>, <class '<a href="../expressions.html#In">sqlglot.expressions.In</a>'>, <class '<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>'>, <class '<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>'>, <class '<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>'>, <class '<a href="../expressions.html#GT">sqlglot.expressions.GT</a>'>}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
@ -3289,7 +3289,7 @@ prefix are statically known.</p>
|
|||
<section id="JOINS">
|
||||
<div class="attr variable">
|
||||
<span class="name">JOINS</span> =
|
||||
<span class="default_value">{('RIGHT', ''), ('', 'INNER'), ('RIGHT', 'OUTER'), ('', '')}</span>
|
||||
<span class="default_value">{('RIGHT', 'OUTER'), ('', 'INNER'), ('RIGHT', ''), ('', '')}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
27199
docs/sqlglot/parser.html
27199
docs/sqlglot/parser.html
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
515
posts/onboarding.md
Normal file
515
posts/onboarding.md
Normal file
|
@ -0,0 +1,515 @@
|
|||
# Table of Contents
|
||||
- [Table of Contents](#table-of-contents)
|
||||
- [Onboarding](#onboarding)
|
||||
- [Tokenizer](#tokenizer)
|
||||
- [Parser](#parser)
|
||||
- [Token Index](#token-index)
|
||||
- [Matching text](#matching-text)
|
||||
- [Utility methods](#utility-methods)
|
||||
- [Command fallback](#command-fallback)
|
||||
- [Expression parsing](#expression-parsing)
|
||||
- [Generator](#generator)
|
||||
- [Generating queries](#generating-queries)
|
||||
- [Utility methods](#utility-methods-1)
|
||||
- [Pretty print](#pretty-print)
|
||||
- [Schema](#schema)
|
||||
- [Optimizer](#optimizer)
|
||||
- [Optimization rules](#optimization-rules)
|
||||
- [Qualify](#qualify)
|
||||
- [Identifier normalization](#identifier-normalization)
|
||||
- [Qualifying tables \& columns](#qualifying-tables--columns)
|
||||
- [Type annotation](#type-annotation)
|
||||
- [Dialects](#dialects)
|
||||
- [Implementing a custom dialect](#implementing-a-custom-dialect)
|
||||
- [Column Level Lineage](#column-level-lineage)
|
||||
- [Implementation details](#implementation-details)
|
||||
|
||||
## Onboarding
|
||||
This document aims to familiarize the reader with SQLGlot's codebase & architecture.
|
||||
|
||||
Generally, some background knowledge about programming languages / compilers is required. A good starting point is [Crafting Interpreters](https://craftinginterpreters.com/) by Robert Nystrom, which served as the foundation when SQLGlot was initially created.
|
||||
|
||||
At a high level, SQLGlot transpilation involves three modules:
|
||||
|
||||
- [Tokenizer](#tokenizer): Converts raw code into a sequence of “tokens”, one for each word/symbol
|
||||
- [Parser](#parser): Converts sequence of tokens into an abstract syntax tree representing the semantics of the raw code
|
||||
- [Generator](#generator): Converts an abstract syntax tree into SQL code
|
||||
|
||||
SQLGlot can transpile SQL between different _SQL dialects_, which are usually associated with different database systems.
|
||||
|
||||
Each dialect has unique syntax that is implemented by overriding the base versions of the three modules. A dialect definition may override any or all of the three base modules.
|
||||
|
||||
The base versions of the modules implement the `sqlglot` dialect (sometimes referred to as "base dialect"), which is designed to accommodate as many common syntax elements as possible across the other supported dialects, thus preventing code duplication.
|
||||
|
||||
SQLGlot includes other modules which are not required for basic transpilation, such as the [Optimizer](#optimizer) and Executor. The Optimizer modifies an abstract syntax tree for other uses (e.g., inferring column-level lineage). The Executor runs SQL code in Python, but its engine is still in an experimental stage, so some functionality may be missing.
|
||||
|
||||
The rest of this document describes the three base modules, dialect-specific overrides, and the Optimizer module.
|
||||
|
||||
## Tokenizer
|
||||
The tokenizer module (`tokens.py`) is responsible for breaking down SQL code into tokens (like keywords, identifiers, etc.), which are the smallest units of meaningful information. This process is also known as lexing/lexical analysis.
|
||||
|
||||
Python is not designed for maximum processing speed/performance, so SQLGlot maintains an equivalent [Rust version of the tokenizer](https://tobikodata.com/sqlglot-jumps-on-the-rust-bandwagon.html) in `sqlglotrs/tokenizer.rs` for improved performance.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Changes in the tokenization logic must be reflected in both the Python and the Rust tokenizer. The goal is for the two implementations to be similar so that there is less cognitive effort to port code between them.
|
||||
|
||||
This example demonstrates the results of using the Tokenizer to tokenize the SQL query `SELECT b FROM table WHERE c = 1`:
|
||||
|
||||
```python
|
||||
from sqlglot import tokenize
|
||||
|
||||
tokens = tokenize("SELECT b FROM table WHERE c = 1")
|
||||
for token in tokens:
|
||||
print(token)
|
||||
|
||||
# <Token token_type: <TokenType.SELECT: 'SELECT'>, text: 'SELECT', line: 1, col: 6, start: 0, end: 5, comments: []>
|
||||
# <Token token_type: <TokenType.VAR: 'VAR'>, text: 'b', line: 1, col: 8, start: 7, end: 7, comments: []>
|
||||
# <Token token_type: <TokenType.FROM: 'FROM'>, text: 'FROM', line: 1, col: 13, start: 9, end: 12, comments: []>
|
||||
# <Token token_type: <TokenType.TABLE: 'TABLE'>, text: 'table', line: 1, col: 19, start: 14, end: 18, comments: []>
|
||||
# <Token token_type: <TokenType.WHERE: 'WHERE'>, text: 'WHERE', line: 1, col: 25, start: 20, end: 24, comments: []>
|
||||
# <Token token_type: <TokenType.VAR: 'VAR'>, text: 'c', line: 1, col: 27, start: 26, end: 26, comments: []>
|
||||
# <Token token_type: <TokenType.EQ: 'EQ'>, text: '=', line: 1, col: 29, start: 28, end: 28, comments: []>
|
||||
# <Token token_type: <TokenType.NUMBER: 'NUMBER'>, text: '1', line: 1, col: 31, start: 30, end: 30, comments: []>
|
||||
```
|
||||
|
||||
The Tokenizer scans the query and converts groups of symbols (or _lexemes_), such as words (e.g., `b`) and operators (e.g., `=`), into tokens. For example, the `VAR` token is used to represent the identifier `b`, whereas the `EQ` token is used to represent the "equals" operator.
|
||||
|
||||
Each token contains information such as its type (`token_type`), the lexeme (`text`) it encapsulates and other metadata such as the lexeme's line (`line`) and column (`col`), which are used to accurately report errors.
|
||||
|
||||
SQLGlot’s `TokenType` enum provides an indirection layer between lexemes and their types. For example, `!=` and `<>` are often used interchangeably to represent "not equals", so SQLGlot groups them together by mapping them both to `TokenType.NEQ`.
|
||||
|
||||
The `Tokenizer.KEYWORDS` and `Tokenizer.SINGLE_TOKENS` are two important dictionaries that map lexemes to the corresponding `TokenType` enum values.
|
||||
|
||||
## Parser
|
||||
The parser module takes the list of tokens generated by the tokenizer and builds an [abstract syntax tree (AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) from them.
|
||||
|
||||
Generally, an AST stores the _meaningful_ constituent components of a SQL statement. For example, to represent a `SELECT` query we can only store its projections, filters, aggregation expressions and so on, but the `SELECT`, `WHERE` and `GROUP BY` keywords don't need to be preserved, since they are implied.
|
||||
|
||||
In SQLGlot's case, the AST usually stores information that captures the _semantics_ of an expression, i.e. its meaning, which is what allows it to transpile SQL code between different dialects. A blog post that explains this idea at a high-level can [be found here](https://tobikodata.com/transpiling_sql1.html).
|
||||
|
||||
The parser processes the tokens corresponding to a SQL statement sequentially and produces an AST to represent it. For example, if a statement’s first token is `CREATE`, the parser will determine what is being created by examining the next token (which could be `SCHEMA`, `TABLE`, `VIEW`, or something else).
|
||||
|
||||
Each semantic concept is represented by an _expression_ in the AST. Converting tokens to expressions is the primary task of the parser.
|
||||
|
||||
As the AST is a core concept of SQLGlot, a more detailed tutorial on its representation, traversal, and transformation can [be found here](https://github.com/tobymao/sqlglot/blob/main/posts/ast_primer.md).
|
||||
|
||||
The next sections describe how the parser processes a SQL statement’s list of tokens, followed by an explanation of how dialect-specific parsing works.
|
||||
|
||||
### Token Index
|
||||
The parser processes a token list sequentially, and maintains an index/cursor that points to the next token to be consumed.
|
||||
|
||||
Once the current token has been successfully processed, the parser moves to the next token in the sequence by calling `_advance()` to increment the index.
|
||||
|
||||
If a token has been consumed but could not be parsed, the parser can move backward to the previous token by calling `_retreat()` to decrement the index.
|
||||
|
||||
In some scenarios, the parser’s behavior depends on both the current and surrounding tokens. The parser can “peek” at the previous and next tokens without consuming them by accessing the `_prev` and `_next` properties, respectively.
|
||||
|
||||
### Matching tokens and text
|
||||
When parsing a SQL statement, the parser must identify specific sets of keywords to correctly build the AST. It does so by “matching” sets of tokens or keywords to infer the structure of the statement.
|
||||
|
||||
For example, these matches occur when parsing the window specification `SUM(x) OVER (PARTITION BY y)`:
|
||||
1. The `SUM` and `(` tokens are matched and `x` is parsed as an identifier
|
||||
2. The `)` token is matched
|
||||
2. The `OVER` and `(` tokens are matched
|
||||
2. The `PARTITION` and `BY` tokens are matched
|
||||
3. The partition clause `y` is parsed
|
||||
4. The `)` token is matched
|
||||
|
||||
The family of `_match` methods perform different varieties of matching. They return `True` and advance the index if there is a match, or return `None` if the tokens do not match.
|
||||
|
||||
The most commonly used `_match` methods are:
|
||||
|
||||
Method | Use
|
||||
-------------------------|-----
|
||||
**`_match(type)`** | Attempt to match a single token with a specific `TokenType`
|
||||
**`_match_set(types)`** | Attempt to match a single token in a set of `TokenType`s
|
||||
**`_match_text_seq(texts)`** | Attempt to match a continuous sequence of strings/texts
|
||||
**`_match_texts(texts)`** | Attempt to match a keyword in a set of strings/texts
|
||||
|
||||
### `_parse` methods
|
||||
SQLGlot’s parser follows the “recursive descent” approach, meaning that it relies on mutually recursive methods in order to understand SQL syntax and the various precedence levels between combined SQL expressions.
|
||||
|
||||
For example, the SQL statement `CREATE TABLE a (col1 INT)` will be parsed into an `exp.Create` expression that includes the kind of object being created (table) and its schema (table name, column names and types). The schema will be parsed into an `exp.Schema` node that contains one column definition `exp.ColumnDef` node for each column.
|
||||
|
||||
The relationships between SQL constructs are encoded in methods whose names begin with “_parse_”, such as `_parse_create()`. We refer to these as “parsing methods”.
|
||||
|
||||
For instance, to parse the statement above the entrypoint method `_parse_create()` is called. Then:
|
||||
- In `_parse_create()`, the parser determines that a `TABLE` is being created
|
||||
- Because a table is being created, the table name is expected next and thus `_parse_table_parts()` is invoked; This is because the table name may have multiple parts such as `catalog.schema.table`
|
||||
- Having processed the table name, the parser continues by parsing the schema definition using `_parse_schema()`
|
||||
|
||||
A key aspect of the SQLGlot parser is that the parsing methods are composable and can be passed as arguments. We describe how that works in the next section.
|
||||
|
||||
### Utility methods
|
||||
SQLGlot provides helper methods for common parsing tasks. They should be used whenever possible to reduce code duplication and manual token/text matching.
|
||||
|
||||
Many helper methods take a parsing method argument, which is invoked to parse part(s) of the clause they're supposed to parse.
|
||||
|
||||
For example, these helper methods parse collections of SQL objects like `col1, col2`, `(PARTITION BY y)`, and `(colA TEXT, colB INT)`, respectively:
|
||||
|
||||
Method | Use
|
||||
-------------------------|-----
|
||||
**`_parse_csv(parse_method)`** | Attempt to parse comma-separated values using the appropriate `parse_method` callable
|
||||
**`_parse_wrapped(parse_method)`** | Attempt to parse a specific construct that is (optionally) wrapped in parentheses
|
||||
**`_parse_wrapped_csv(parse_method)`** | Attempt to parse comma-separated values that are (optionally) wrapped in parentheses
|
||||
|
||||
|
||||
### Command fallback
|
||||
The SQL language specification and dialect variations are vast, and SQLGlot cannot parse every possible statement.
|
||||
|
||||
Instead of erroring on statements it cannot parse, SQLGlot falls back to storing the code that cannot be parsed in an `exp.Command` expression.
|
||||
|
||||
This allows SQLGlot to return the code unmodified even though it cannot parse it. Because the code is unmodified, any dialect-specific components will not be transpiled.
|
||||
|
||||
### Dialect-specific parsing
|
||||
The base parser’s goal is to represent as many common constructs from different SQL dialects as possible. This makes the parser more lenient and less-repetitive/concise.
|
||||
|
||||
> [!WARNING]
|
||||
> SQLGlot does not aim to be a SQL validator, so it may not detect certain syntax errors.
|
||||
|
||||
Dialect-specific parser behavior is implemented in two ways: feature flags and parser overrides.
|
||||
|
||||
If two different parsing behaviors are common across dialects, the base parser may implement both and use feature flags to determine which should be used for a specific dialect. In contrast, parser overrides directly replace specific base parser methods.
|
||||
|
||||
Therefore, each dialect’s Parser class may specify:
|
||||
|
||||
- Feature flags such as `SUPPORTS_IMPLICIT_UNNEST` or `SUPPORTS_PARTITION_SELECTION`, which are used to control the behavior of base parser methods.
|
||||
|
||||
- Sets of tokens that play a similar role, such as `RESERVED_TOKENS` that cannot be used as object names.
|
||||
|
||||
- Sets of `token -> Callable` mappings, implemented with Python lambda functions
|
||||
- When the parser comes across the token (string or `TokenType`) on the left-hand side it invokes the mapping function on the right to create the corresponding Expression / AST node.
|
||||
- The lambda function may return an expression directly or a `_parse_ method` that determines what expression should be returned.
|
||||
- The mappings are grouped by general semantic type (e.g., functions, parsers for `ALTER` statements, etc.), with each type having its own dictionary.
|
||||
|
||||
An example `token -> Callable` mapping is the `FUNCTIONS` dictionary that builds the appropriate `exp.Func` node based on the string key:
|
||||
|
||||
```Python3
|
||||
FUNCTIONS: t.Dict[str, t.Callable] = {
|
||||
"LOG2": lambda args: exp.Log(this=exp.Literal.number(2), expression=seq_get(args, 0)),
|
||||
"LOG10": lambda args: exp.Log(this=exp.Literal.number(10), expression=seq_get(args, 0)),
|
||||
"MOD": build_mod,
|
||||
…,
|
||||
}
|
||||
```
|
||||
|
||||
In this dialect, the `LOG2()` function calculates a logarithm of base 2, which is represented in SQLGlot by the general `exp.Log` expression.
|
||||
|
||||
The logarithm base and the user-specified value to `log` are stored in the node's arguments. The former is stored as an integer literal in the `this` argument, and the latter is stored in the `expression` argument.
|
||||
|
||||
## Generator
|
||||
After the parser has created an AST, the generator module is responsible for converting it back into SQL code.
|
||||
|
||||
Each dialect’s Generator class may specify:
|
||||
|
||||
- A set of flags such as `ALTER_TABLE_INCLUDE_COLUMN_KEYWORD`. As with the parser, these flags control the behavior of base Generator methods.
|
||||
|
||||
- Set of `Expression -> str` mappings
|
||||
- The generator traverses the AST recursively and produces the equivalent string for each expression it visits. This can be thought of as the reverse operation of the parser, where expressions / AST nodes are converted back to strings.
|
||||
- The mappings are grouped by general semantic type (e.g., data type expressions), with each type having its own dictionary.
|
||||
|
||||
The `Expression -> str` mappings can be implemented as either:
|
||||
- Entries in the `TRANSFORMS` dictionary, which are best suited for single-line generations.
|
||||
- Functions with names of the form `<expr_name>_sql(...)`. For example, to generate SQL for an `exp.SessionParameter` node we can override the base generator method by defining the method `def sessionparameter_sql(...)` in a dialect’s Generator class.
|
||||
|
||||
### Generating queries
|
||||
The key method in the Generator module is the `sql()` function, which is responsible for generating the string representation of each expression.
|
||||
|
||||
First, it locates the mapping of expression to string or generator `Callable` by examining the keys of the `TRANSFORMS` dictionary and searching the `Generator`'s methods for an `<exprname>_sql()` method.
|
||||
|
||||
It then calls the appropriate callable to produce the corresponding SQL code.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> If there is both a `TRANSFORM` entry and an `<expr_name>_sql(...)` method for a given expression, the generator will use the former to convert the expression into a string.
|
||||
|
||||
### Utility methods
|
||||
The Generator defines helper methods containing quality-of-life abstractions over the `sql()` method, such as:
|
||||
|
||||
Method | Use
|
||||
-------------------------|-----
|
||||
**`expressions()`** | Generates the string representation for a list of `Expression`s, employing a series of arguments to help with their formatting
|
||||
**`func()`, `rename_func()`** | Given a function name and an `Expression`, returns the string representation of a function call by generating the expression's args as the func args.
|
||||
|
||||
These methods should be used whenever possible to reduce code duplication.
|
||||
|
||||
### Pretty print
|
||||
Pretty printing refers to the process of generating formatted and consistently styled SQL, which improves readability, debugging, and code reviewing.
|
||||
|
||||
SQLGlot generates an AST’s SQL code on a single line by default, but users may specify that it be `pretty` and include new lines and indenting.
|
||||
|
||||
It is up to the developer to ensure that whitespaces and newlines are embedded in the SQL correctly. To aid in that process, the Generator class provides the `sep()` and `seg()` helper methods.
|
||||
|
||||
## Dialects
|
||||
We briefly mentioned dialect-specific behaviors while describing the SQLGlot base Tokenizer, Parser, and Generator modules. This section expands on how to specify a new SQL dialect.
|
||||
|
||||
As [described above](#dialect-specific-parsing), SQLGlot attempts to bridge dialect-specific variations in a "superset" dialect, which we refer to as the _sqlglot_ dialect.
|
||||
|
||||
All other SQL dialects have their own Dialect subclass whose Tokenizer, Parser and Generator components extend or override the base modules as needed.
|
||||
|
||||
The base dialect components are defined in `tokens.py`, `parser.py` and `generator.py`. Dialect definitions can be found under `dialects/<dialect>.py`.
|
||||
|
||||
When adding features that will be used by multiple dialects, it's best to place the common/overlapping parts in the base _sqlglot_ dialect to prevent code repetition across other dialects. Any dialect-specific features that cannot (or should not) be repeated can be specified in the dialect’s subclass.
|
||||
|
||||
The Dialect class contains flags that are visible to its Tokenizer, Parser and Generator components. Flags are only added in a Dialect when they need to be visible to at least two components, in order to avoid code duplication.
|
||||
|
||||
### Implementing a custom dialect
|
||||
Creating a new SQL dialect may seem complicated at first, but it is actually quite simple in SQLGlot.
|
||||
|
||||
This example demonstrates defining a new dialect named `Custom` that extends or overrides components of the base Dialect modules:
|
||||
|
||||
```Python
|
||||
from sqlglot import exp
|
||||
from sqlglot.dialects.dialect import Dialect
|
||||
from sqlglot.generator import Generator
|
||||
from sqlglot.tokens import Tokenizer, TokenType
|
||||
|
||||
|
||||
class Custom(Dialect):
|
||||
class Tokenizer(Tokenizer):
|
||||
QUOTES = ["'", '"'] # Strings can be delimited by either single or double quotes
|
||||
IDENTIFIERS = ["`"] # Identifiers can be delimited by backticks
|
||||
|
||||
# Associates certain meaningful words with tokens that capture their intent
|
||||
KEYWORDS = {
|
||||
**Tokenizer.KEYWORDS,
|
||||
"INT64": TokenType.BIGINT,
|
||||
"FLOAT64": TokenType.DOUBLE,
|
||||
}
|
||||
|
||||
class Parser(Parser):
|
||||
# Specifies how certain tokens are mapped to function AST nodes
|
||||
FUNCTIONS = {
|
||||
**parser.Parser.FUNCTIONS,
|
||||
"APPROX_PERCENTILE": exp.ApproxQuantile.from_arg_list,
|
||||
}
|
||||
|
||||
# Specifies how a specific construct e.g. CONSTRAINT is parsed
|
||||
def _parse_constraint(self) -> t.Optional[exp.Expression]:
|
||||
return super()._parse_constraint() or self._parse_projection_def()
|
||||
|
||||
class Generator(Generator):
|
||||
# Specifies how AST nodes, i.e. subclasses of exp.Expression, should be converted into SQL
|
||||
TRANSFORMS = {
|
||||
exp.Array: lambda self, e: f"[{self.expressions(e)}]",
|
||||
}
|
||||
|
||||
# Specifies how AST nodes representing data types should be converted into SQL
|
||||
TYPE_MAPPING = {
|
||||
exp.DataType.Type.TINYINT: "INT64",
|
||||
exp.DataType.Type.SMALLINT: "INT64",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Even though this example is a fairly realistic starting point, we strongly encourage you to study existing dialect implementations to understand how their various components can be modified.
|
||||
|
||||
## Schema
|
||||
Previous sections described the SQLGlot components used for transpilation. This section and the next describe components needed to implement other SQLGlot functionality like column-level lineage.
|
||||
|
||||
A Schema represents the structure of a database schema, including the tables/views it contains and their column names and data types.
|
||||
|
||||
The file `schema.py` defines the abstract classes `Schema` and `AbstractMappingSchema`; the implementing class is `MappingSchema`.
|
||||
|
||||
A `MappingSchema` object is defined with a nested dictionary, as in this example:
|
||||
|
||||
```Python
|
||||
schema = MappingSchema({"t": {"x": "int", "y": "datetime"}})
|
||||
```
|
||||
|
||||
The keys of the top-level dictionary are table names (`t`), the keys of `t`’s dictionary are its column names (`x` and `y`), and the values are the column data types (`int` and `datetime`).
|
||||
|
||||
A schema provides information required by other SQLGlot modules (most importantly, the optimizer and column level lineage).
|
||||
|
||||
Schema information is used to enrich ASTs with actions like replacing stars (`SELECT *`) with the names of the columns being selected from the upstream tables.
|
||||
|
||||
## Optimizer
|
||||
The optimizer module in SQLGlot (`optimizer.py`) is responsible for producing canonicalized and efficient SQL queries by applying a series of optimization rules.
|
||||
|
||||
Optimizations include simplifying expressions, removing redundant operations, and rewriting queries for better performance.
|
||||
|
||||
The optimizer operates on the abstract syntax tree (AST) returned by the parser, transforming it into a more compact form while preserving the semantics of the original query.
|
||||
|
||||
> [!NOTE]
|
||||
> The optimizer performs only logical optimization; The underlying engine is almost always better at optimizing for performance.
|
||||
|
||||
### Optimization rules
|
||||
The optimizer essentially applies [a list of rules](https://sqlglot.com/sqlglot/optimizer.html) in a sequential manner, where each rule takes in an AST and produces a canonicalized and optimized version of it.
|
||||
|
||||
> [!WARNING]
|
||||
> The rule order is important, as some rules depend on others to work properly. We discourage manually applying individual rules, which may result in erroneous or unpredictable behavior.
|
||||
|
||||
The first, foundational optimization task is to _standardize_ an AST, which is required by other optimization steps. These rules implement canonicalization:
|
||||
|
||||
#### Qualify
|
||||
The most important rule in the optimizer is `qualify`, which is responsible for rewriting the AST such that all tables and columns are _normalized_ and _qualified_.
|
||||
|
||||
##### Identifier normalization
|
||||
The normalization step (`normalize_identifiers.py`) transforms some identifiers by converting them into lower or upper case, ensuring the semantics are preserved in each case (e.g. by respecting case-sensitivity).
|
||||
|
||||
This transformation reflects how identifiers would be resolved by the engine corresponding to each SQL dialect, and plays a very important role in the standardization of the AST.
|
||||
|
||||
> [!NOTE]
|
||||
> Some dialects (e.g. DuckDB) treat identifiers as case-insensitive even when they're quoted, so all identifiers are normalized.
|
||||
|
||||
Different dialects may have different normalization strategies. For instance, in Snowflake unquoted identifiers are normalized to uppercase.
|
||||
|
||||
This example demonstrates how the identifier `Foo` is normalized to lowercase `foo` in the default sqlglot dialect and uppercase `FOO` in the Snowflake dialect:
|
||||
|
||||
```Python
|
||||
import sqlglot
|
||||
from sqlglot.optimizer.normalize_identifiers import normalize_identifiers
|
||||
|
||||
normalize_identifiers("Foo").sql()
|
||||
# 'foo'
|
||||
|
||||
normalize_identifiers("Foo", dialect="snowflake").sql("snowflake")
|
||||
# 'FOO'
|
||||
```
|
||||
|
||||
##### Qualifying tables and columns
|
||||
After normalizing the identifiers, all table and column names are qualified so there is no ambiguity about the data sources referenced by the query.
|
||||
|
||||
This means that all table references are assigned an alias and all column names are prefixed with their source table’s name.
|
||||
|
||||
This example demonstrates qualifying the query `SELECT col FROM tbl`, where `tbl` contains a single column `col`. Note that the table’s schema is passed as an argument to `qualify()`:
|
||||
|
||||
```Python
|
||||
import sqlglot
|
||||
from sqlglot.optimizer.qualify import qualify
|
||||
|
||||
schema = {"tbl": {"col": "INT"}}
|
||||
expression = sqlglot.parse_one("SELECT col FROM tbl")
|
||||
qualify(expression, schema=schema).sql()
|
||||
|
||||
# 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'
|
||||
```
|
||||
|
||||
The `qualify` rule also offers a set of sub-rules that further canonicalize the query.
|
||||
|
||||
For instance, `qualify()` can expand stars such that each `SELECT *` is replaced by a projection list of the selected columns.
|
||||
|
||||
This example modifies the previous example’s query to use `SELECT *`. `qualify()` expands the star and returns the same output as before:
|
||||
|
||||
```Python
|
||||
import sqlglot
|
||||
from sqlglot.optimizer.qualify import qualify
|
||||
|
||||
|
||||
schema = {"tbl": {"col": "INT"}}
|
||||
expression = sqlglot.parse_one("SELECT * FROM tbl")
|
||||
qualify(expression, schema=schema).sql()
|
||||
|
||||
|
||||
# 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'
|
||||
```
|
||||
|
||||
#### Type Inference
|
||||
Some SQL operators’ behavior changes based on the data type of its inputs.
|
||||
|
||||
For example, in some SQL dialects `+` can be used for either adding numbers or concatenating strings. The database uses its knowledge of column data types to determine which operation should be executed in a specific situation.
|
||||
|
||||
Like the database, SQLGlot needs to know column data types to perform some optimizations. In many cases, both transpilation and optimization need type information to work properly.
|
||||
|
||||
Type annotation begins with user-provided information about column data types for at least one table. SQLGlot then traverses the AST, propagating that type information and attempting to infer the type returned by each AST expression.
|
||||
|
||||
Users may need to provide column type information for multiple tables to successfully annotate all expressions in the AST.
|
||||
|
||||
## Column Level Lineage
|
||||
Column-level lineage (CLL) traces the flow of data from column to column as tables select and modify columns from one another in a SQL codebase.
|
||||
|
||||
CLL provides critical information about data lineage that helps in understanding how data is derived, transformed, and used across different parts of a data pipeline or database system.
|
||||
|
||||
SQLGlot’s CLL implementation is defined in `lineage.py`. It operates on a collection of queries that `SELECT` from one another. It assumes that the graph/network of queries forms a directed acyclic graph (DAG) where no two tables `SELECT` from each other.
|
||||
|
||||
The user must provide:
|
||||
A “target” query whose upstream column lineage should be traced
|
||||
The queries linking all tables upstream of the target query
|
||||
The schemas (column names and types) of the root tables in the DAG
|
||||
|
||||
In this example, we trace the lineage of the column `traced_col`:
|
||||
|
||||
```Python
|
||||
from sqlglot.lineage import lineage
|
||||
|
||||
target_query = “””
|
||||
WITH cte AS (
|
||||
SELECT
|
||||
traced_col
|
||||
FROM
|
||||
intermediate_table
|
||||
)
|
||||
SELECT
|
||||
traced_col
|
||||
FROM
|
||||
cte
|
||||
“””
|
||||
|
||||
node = lineage(
|
||||
column="traced_col",
|
||||
sql=target_query,
|
||||
sources={"intermediate_table": "SELECT * FROM root_table"},
|
||||
schema={"root_table": {"traced_col": "int"}},
|
||||
)
|
||||
```
|
||||
|
||||
The `lineage()` function’s `sql` argument takes the target query, the `sources` argument takes a dictionary of source table names and the query that produces each table, and the `schema` argument takes the column names/types of the graph’s root tables.
|
||||
|
||||
### Implementation details
|
||||
This section describes how SQLGlot traces the lineage in the previous example.
|
||||
|
||||
Before lineage can be traced, the following preparatory steps must be carried out:
|
||||
|
||||
1. Replace the `sql` query’s table references with their `sources`’ queries, such that we have a single standalone query.
|
||||
|
||||
This example demonstrates how the earlier example’s reference to `intermediate_table` inside the CTE is replaced with the query that generated it, `SELECT * FROM root_table`, and given the alias `intermediate_table`:
|
||||
|
||||
```SQL
|
||||
WITH cte AS (
|
||||
SELECT
|
||||
traced_col
|
||||
FROM (
|
||||
SELECT
|
||||
*
|
||||
FROM
|
||||
root_table
|
||||
) AS intermediate_table
|
||||
)
|
||||
SELECT
|
||||
traced_col
|
||||
FROM
|
||||
cte
|
||||
```
|
||||
|
||||
2. Qualify the query produced by the earlier step, expanding `*`s and qualifying all column references with the corresponding source tables:
|
||||
|
||||
```SQL
|
||||
WITH cte AS (
|
||||
SELECT
|
||||
intermediate_table.traced_col
|
||||
FROM (
|
||||
SELECT
|
||||
root_table.traced_col
|
||||
FROM
|
||||
root_table AS root_table
|
||||
) AS intermediate_table
|
||||
)
|
||||
SELECT
|
||||
cte.traced_col
|
||||
FROM
|
||||
cte AS cte
|
||||
```
|
||||
|
||||
After these operations, the query is canonicalized and the lineage can be traced.
|
||||
|
||||
Tracing is a top-down process, meaning that the search starts from `cte.traced_col` (outer scope) and traverse inwards to find that its origin is `intermediate_table.traced_col`, which in turn is based on `root_table.traced_col`.
|
||||
|
||||
The search continues until the innermost scope has been visited. At that point all `sources` are exhausted, so the `schema` is searched to validate that the root table (`root_table` in our example) exists and that the target column `traced_col` is defined in it.
|
||||
|
||||
At each step, the column of interest (i.e., `cte.traced_col`, `intermediate_table.traced_col`. etc) is wrapped in a lineage object `Node` that is linked to its upstream nodes.
|
||||
|
||||
This approach incrementally builds a linked list of `Nodes` that traces the lineage for the target column: `root_table.traced_col -> intermediate_table.traced_col -> cte.traced_col`.
|
||||
|
||||
The `lineage()` output can be visualized using the `node.to_html()` function, which generates an HTML lineage graph using `vis.js`.
|
||||
|
||||

|
BIN
posts/onboarding_images/lineage_img.png
Normal file
BIN
posts/onboarding_images/lineage_img.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 188 KiB |
2
setup.py
2
setup.py
|
@ -34,10 +34,12 @@ setup(
|
|||
"pandas",
|
||||
"pandas-stubs",
|
||||
"python-dateutil",
|
||||
"pytz",
|
||||
"pdoc",
|
||||
"pre-commit",
|
||||
"ruff==0.4.3",
|
||||
"types-python-dateutil",
|
||||
"types-pytz",
|
||||
"typing_extensions",
|
||||
"maturin>=1.4,<2.0",
|
||||
],
|
||||
|
|
|
@ -6,6 +6,12 @@ from sqlglot.tokens import TokenType
|
|||
|
||||
|
||||
class Athena(Trino):
|
||||
class Tokenizer(Trino.Tokenizer):
|
||||
KEYWORDS = {
|
||||
**Trino.Tokenizer.KEYWORDS,
|
||||
"UNLOAD": TokenType.COMMAND,
|
||||
}
|
||||
|
||||
class Parser(Trino.Parser):
|
||||
STATEMENT_PARSERS = {
|
||||
**Trino.Parser.STATEMENT_PARSERS,
|
||||
|
|
|
@ -252,6 +252,7 @@ class BigQuery(Dialect):
|
|||
TIME_MAPPING = {
|
||||
"%D": "%m/%d/%y",
|
||||
"%E6S": "%S.%f",
|
||||
"%e": "%-d",
|
||||
}
|
||||
|
||||
FORMAT_MAPPING = {
|
||||
|
@ -401,6 +402,9 @@ class BigQuery(Dialect):
|
|||
),
|
||||
"TIMESTAMP_SECONDS": lambda args: exp.UnixToTime(this=seq_get(args, 0)),
|
||||
"TO_JSON_STRING": exp.JSONFormat.from_arg_list,
|
||||
"FORMAT_DATETIME": lambda args: exp.TimeToStr(
|
||||
this=exp.TsOrDsToTimestamp(this=seq_get(args, 1)), format=seq_get(args, 0)
|
||||
),
|
||||
}
|
||||
|
||||
FUNCTION_PARSERS = {
|
||||
|
@ -500,7 +504,7 @@ class BigQuery(Dialect):
|
|||
table.set("db", exp.Identifier(this=parts[0]))
|
||||
table.set("this", exp.Identifier(this=parts[1]))
|
||||
|
||||
if any("." in p.name for p in table.parts):
|
||||
if isinstance(table.this, exp.Identifier) and any("." in p.name for p in table.parts):
|
||||
catalog, db, this, *rest = (
|
||||
exp.to_identifier(p, quoted=True)
|
||||
for p in split_num_words(".".join(p.name for p in table.parts), ".", 3)
|
||||
|
@ -583,6 +587,28 @@ class BigQuery(Dialect):
|
|||
|
||||
return bracket
|
||||
|
||||
def _parse_unnest(self, with_alias: bool = True) -> t.Optional[exp.Unnest]:
|
||||
unnest = super()._parse_unnest(with_alias=with_alias)
|
||||
|
||||
if not unnest:
|
||||
return None
|
||||
|
||||
unnest_expr = seq_get(unnest.expressions, 0)
|
||||
if unnest_expr:
|
||||
from sqlglot.optimizer.annotate_types import annotate_types
|
||||
|
||||
unnest_expr = annotate_types(unnest_expr)
|
||||
|
||||
# Unnesting a nested array (i.e array of structs) explodes the top-level struct fields,
|
||||
# in contrast to other dialects such as DuckDB which flattens only the array by default
|
||||
if unnest_expr.is_type(exp.DataType.Type.ARRAY) and any(
|
||||
array_elem.is_type(exp.DataType.Type.STRUCT)
|
||||
for array_elem in unnest_expr._type.expressions
|
||||
):
|
||||
unnest.set("explode_array", True)
|
||||
|
||||
return unnest
|
||||
|
||||
class Generator(generator.Generator):
|
||||
EXPLICIT_SET_OP = True
|
||||
INTERVAL_ALLOWS_PLURAL_FORM = False
|
||||
|
@ -606,6 +632,7 @@ class BigQuery(Dialect):
|
|||
NAMED_PLACEHOLDER_TOKEN = "@"
|
||||
HEX_FUNC = "TO_HEX"
|
||||
WITH_PROPERTIES_PREFIX = "OPTIONS"
|
||||
SUPPORTS_EXPLODING_PROJECTIONS = False
|
||||
|
||||
TRANSFORMS = {
|
||||
**generator.Generator.TRANSFORMS,
|
||||
|
@ -878,8 +905,16 @@ class BigQuery(Dialect):
|
|||
return super().table_parts(expression)
|
||||
|
||||
def timetostr_sql(self, expression: exp.TimeToStr) -> str:
|
||||
this = expression.this if isinstance(expression.this, exp.TsOrDsToDate) else expression
|
||||
return self.func("FORMAT_DATE", self.format_time(expression), this.this)
|
||||
if isinstance(expression.this, exp.TsOrDsToTimestamp):
|
||||
func_name = "FORMAT_DATETIME"
|
||||
else:
|
||||
func_name = "FORMAT_DATE"
|
||||
this = (
|
||||
expression.this
|
||||
if isinstance(expression.this, (exp.TsOrDsToTimestamp, exp.TsOrDsToDate))
|
||||
else expression
|
||||
)
|
||||
return self.func(func_name, self.format_time(expression), this.this)
|
||||
|
||||
def eq_sql(self, expression: exp.EQ) -> str:
|
||||
# Operands of = cannot be NULL in BigQuery
|
||||
|
|
|
@ -1,10 +1,12 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import typing as t
|
||||
import datetime
|
||||
|
||||
from sqlglot import exp, generator, parser, tokens
|
||||
from sqlglot.dialects.dialect import (
|
||||
Dialect,
|
||||
NormalizationStrategy,
|
||||
arg_max_or_min_no_count,
|
||||
build_date_delta,
|
||||
build_formatted_time,
|
||||
|
@ -31,7 +33,7 @@ def _build_date_format(args: t.List) -> exp.TimeToStr:
|
|||
|
||||
timezone = seq_get(args, 2)
|
||||
if timezone:
|
||||
expr.set("timezone", timezone)
|
||||
expr.set("zone", timezone)
|
||||
|
||||
return expr
|
||||
|
||||
|
@ -104,6 +106,28 @@ def _datetime_delta_sql(name: str) -> t.Callable[[Generator, DATEΤΙΜΕ_DELTA]
|
|||
return _delta_sql
|
||||
|
||||
|
||||
def _timestrtotime_sql(self: ClickHouse.Generator, expression: exp.TimeStrToTime):
|
||||
tz = expression.args.get("zone")
|
||||
datatype = exp.DataType.build(exp.DataType.Type.TIMESTAMP)
|
||||
ts = expression.this
|
||||
if tz:
|
||||
# build a datatype that encodes the timezone as a type parameter, eg DateTime('America/Los_Angeles')
|
||||
datatype = exp.DataType.build(
|
||||
exp.DataType.Type.TIMESTAMPTZ, # Type.TIMESTAMPTZ maps to DateTime
|
||||
expressions=[exp.DataTypeParam(this=tz)],
|
||||
)
|
||||
|
||||
if isinstance(ts, exp.Literal):
|
||||
# strip the timezone out of the literal, eg turn '2020-01-01 12:13:14-08:00' into '2020-01-01 12:13:14'
|
||||
# this is because Clickhouse encodes the timezone as a data type parameter and throws an error if it's part of the timestamp string
|
||||
ts_without_tz = (
|
||||
datetime.datetime.fromisoformat(ts.name).replace(tzinfo=None).isoformat(sep=" ")
|
||||
)
|
||||
ts = exp.Literal.string(ts_without_tz)
|
||||
|
||||
return self.sql(exp.cast(ts, datatype, dialect=self.dialect))
|
||||
|
||||
|
||||
class ClickHouse(Dialect):
|
||||
NORMALIZE_FUNCTIONS: bool | str = False
|
||||
NULL_ORDERING = "nulls_are_last"
|
||||
|
@ -112,10 +136,15 @@ class ClickHouse(Dialect):
|
|||
LOG_BASE_FIRST: t.Optional[bool] = None
|
||||
FORCE_EARLY_ALIAS_REF_EXPANSION = True
|
||||
|
||||
# https://github.com/ClickHouse/ClickHouse/issues/33935#issue-1112165779
|
||||
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE
|
||||
|
||||
UNESCAPED_SEQUENCES = {
|
||||
"\\0": "\0",
|
||||
}
|
||||
|
||||
CREATABLE_KIND_MAPPING = {"DATABASE": "SCHEMA"}
|
||||
|
||||
class Tokenizer(tokens.Tokenizer):
|
||||
COMMENTS = ["--", "#", "#!", ("/*", "*/")]
|
||||
IDENTIFIERS = ['"', "`"]
|
||||
|
@ -424,6 +453,27 @@ class ClickHouse(Dialect):
|
|||
"INDEX",
|
||||
}
|
||||
|
||||
PLACEHOLDER_PARSERS = {
|
||||
**parser.Parser.PLACEHOLDER_PARSERS,
|
||||
TokenType.L_BRACE: lambda self: self._parse_query_parameter(),
|
||||
}
|
||||
|
||||
def _parse_types(
|
||||
self, check_func: bool = False, schema: bool = False, allow_identifiers: bool = True
|
||||
) -> t.Optional[exp.Expression]:
|
||||
dtype = super()._parse_types(
|
||||
check_func=check_func, schema=schema, allow_identifiers=allow_identifiers
|
||||
)
|
||||
if isinstance(dtype, exp.DataType):
|
||||
# Mark every type as non-nullable which is ClickHouse's default. This marker
|
||||
# helps us transpile types from other dialects to ClickHouse, so that we can
|
||||
# e.g. produce `CAST(x AS Nullable(String))` from `CAST(x AS TEXT)`. If there
|
||||
# is a `NULL` value in `x`, the former would fail in ClickHouse without the
|
||||
# `Nullable` type constructor
|
||||
dtype.set("nullable", False)
|
||||
|
||||
return dtype
|
||||
|
||||
def _parse_extract(self) -> exp.Extract | exp.Anonymous:
|
||||
index = self._index
|
||||
this = self._parse_bitwise()
|
||||
|
@ -433,7 +483,7 @@ class ClickHouse(Dialect):
|
|||
|
||||
# We return Anonymous here because extract and regexpExtract have different semantics,
|
||||
# so parsing extract(foo, bar) into RegexpExtract can potentially break queries. E.g.,
|
||||
# `extract('foobar', 'b')` works, but CH crashes for `regexpExtract('foobar', 'b')`.
|
||||
# `extract('foobar', 'b')` works, but ClickHouse crashes for `regexpExtract('foobar', 'b')`.
|
||||
#
|
||||
# TODO: can we somehow convert the former into an equivalent `regexpExtract` call?
|
||||
self._match(TokenType.COMMA)
|
||||
|
@ -454,14 +504,11 @@ class ClickHouse(Dialect):
|
|||
|
||||
return this
|
||||
|
||||
def _parse_placeholder(self) -> t.Optional[exp.Expression]:
|
||||
def _parse_query_parameter(self) -> t.Optional[exp.Expression]:
|
||||
"""
|
||||
Parse a placeholder expression like SELECT {abc: UInt32} or FROM {table: Identifier}
|
||||
https://clickhouse.com/docs/en/sql-reference/syntax#defining-and-using-query-parameters
|
||||
"""
|
||||
if not self._match(TokenType.L_BRACE):
|
||||
return None
|
||||
|
||||
this = self._parse_id_var()
|
||||
self._match(TokenType.COLON)
|
||||
kind = self._parse_types(check_func=False, allow_identifiers=False) or (
|
||||
|
@ -589,7 +636,7 @@ class ClickHouse(Dialect):
|
|||
|
||||
if isinstance(expr, exp.Window):
|
||||
# The window's func was parsed as Anonymous in base parser, fix its
|
||||
# type to be CH style CombinedAnonymousAggFunc / AnonymousAggFunc
|
||||
# type to be ClickHouse style CombinedAnonymousAggFunc / AnonymousAggFunc
|
||||
expr.set("this", func)
|
||||
elif params:
|
||||
# Params have blocked super()._parse_function() from parsing the following window
|
||||
|
@ -715,6 +762,7 @@ class ClickHouse(Dialect):
|
|||
GROUPINGS_SEP = ""
|
||||
SET_OP_MODIFIERS = False
|
||||
SUPPORTS_TABLE_ALIAS_COLUMNS = False
|
||||
VALUES_AS_TABLE = False
|
||||
|
||||
STRING_TYPE_MAPPING = {
|
||||
exp.DataType.Type.CHAR: "String",
|
||||
|
@ -741,7 +789,10 @@ class ClickHouse(Dialect):
|
|||
exp.DataType.Type.ARRAY: "Array",
|
||||
exp.DataType.Type.BIGINT: "Int64",
|
||||
exp.DataType.Type.DATE32: "Date32",
|
||||
exp.DataType.Type.DATETIME: "DateTime",
|
||||
exp.DataType.Type.DATETIME64: "DateTime64",
|
||||
exp.DataType.Type.TIMESTAMP: "DateTime",
|
||||
exp.DataType.Type.TIMESTAMPTZ: "DateTime",
|
||||
exp.DataType.Type.DOUBLE: "Float64",
|
||||
exp.DataType.Type.ENUM: "Enum",
|
||||
exp.DataType.Type.ENUM8: "Enum8",
|
||||
|
@ -790,6 +841,7 @@ class ClickHouse(Dialect):
|
|||
exp.CurrentDate: lambda self, e: self.func("CURRENT_DATE"),
|
||||
exp.DateAdd: _datetime_delta_sql("DATE_ADD"),
|
||||
exp.DateDiff: _datetime_delta_sql("DATE_DIFF"),
|
||||
exp.DateStrToDate: rename_func("toDate"),
|
||||
exp.DateSub: _datetime_delta_sql("DATE_SUB"),
|
||||
exp.Explode: rename_func("arrayJoin"),
|
||||
exp.Final: lambda self, e: f"{self.sql(e, 'this')} FINAL",
|
||||
|
@ -810,8 +862,9 @@ class ClickHouse(Dialect):
|
|||
"position", e.this, e.args.get("substr"), e.args.get("position")
|
||||
),
|
||||
exp.TimeToStr: lambda self, e: self.func(
|
||||
"DATE_FORMAT", e.this, self.format_time(e), e.args.get("timezone")
|
||||
"DATE_FORMAT", e.this, self.format_time(e), e.args.get("zone")
|
||||
),
|
||||
exp.TimeStrToTime: _timestrtotime_sql,
|
||||
exp.TimestampAdd: _datetime_delta_sql("TIMESTAMP_ADD"),
|
||||
exp.TimestampSub: _datetime_delta_sql("TIMESTAMP_SUB"),
|
||||
exp.VarMap: lambda self, e: _lower_func(var_map_sql(self, e)),
|
||||
|
@ -823,6 +876,7 @@ class ClickHouse(Dialect):
|
|||
exp.UnixToTime: _unix_to_time_sql,
|
||||
exp.TimestampTrunc: timestamptrunc_sql(zone=True),
|
||||
exp.Variance: rename_func("varSamp"),
|
||||
exp.SchemaCommentProperty: lambda self, e: self.naked_property(e),
|
||||
exp.Stddev: rename_func("stddevSamp"),
|
||||
}
|
||||
|
||||
|
@ -833,7 +887,7 @@ class ClickHouse(Dialect):
|
|||
exp.OnCluster: exp.Properties.Location.POST_NAME,
|
||||
}
|
||||
|
||||
# there's no list in docs, but it can be found in Clickhouse code
|
||||
# There's no list in docs, but it can be found in Clickhouse code
|
||||
# see `ClickHouse/src/Parsers/ParserCreate*.cpp`
|
||||
ON_CLUSTER_TARGETS = {
|
||||
"DATABASE",
|
||||
|
@ -845,6 +899,14 @@ class ClickHouse(Dialect):
|
|||
"NAMED COLLECTION",
|
||||
}
|
||||
|
||||
# https://clickhouse.com/docs/en/sql-reference/data-types/nullable
|
||||
NON_NULLABLE_TYPES = {
|
||||
exp.DataType.Type.ARRAY,
|
||||
exp.DataType.Type.MAP,
|
||||
exp.DataType.Type.NULLABLE,
|
||||
exp.DataType.Type.STRUCT,
|
||||
}
|
||||
|
||||
def strtodate_sql(self, expression: exp.StrToDate) -> str:
|
||||
strtodate_sql = self.function_fallback_sql(expression)
|
||||
|
||||
|
@ -863,6 +925,14 @@ class ClickHouse(Dialect):
|
|||
|
||||
return super().cast_sql(expression, safe_prefix=safe_prefix)
|
||||
|
||||
def trycast_sql(self, expression: exp.TryCast) -> str:
|
||||
dtype = expression.to
|
||||
if not dtype.is_type(*self.NON_NULLABLE_TYPES, check_nullable=True):
|
||||
# Casting x into Nullable(T) appears to behave similarly to TRY_CAST(x AS T)
|
||||
dtype.set("nullable", True)
|
||||
|
||||
return super().cast_sql(expression)
|
||||
|
||||
def _jsonpathsubscript_sql(self, expression: exp.JSONPathSubscript) -> str:
|
||||
this = self.json_path_part(expression.this)
|
||||
return str(int(this) + 1) if is_int(this) else this
|
||||
|
@ -904,9 +974,30 @@ class ClickHouse(Dialect):
|
|||
#
|
||||
# https://clickhouse.com/docs/en/sql-reference/data-types/string
|
||||
if expression.this in self.STRING_TYPE_MAPPING:
|
||||
return "String"
|
||||
dtype = "String"
|
||||
else:
|
||||
dtype = super().datatype_sql(expression)
|
||||
|
||||
return super().datatype_sql(expression)
|
||||
# This section changes the type to `Nullable(...)` if the following conditions hold:
|
||||
# - It's marked as nullable - this ensures we won't wrap ClickHouse types with `Nullable`
|
||||
# and change their semantics
|
||||
# - It's not the key type of a `Map`. This is because ClickHouse enforces the following
|
||||
# constraint: "Type of Map key must be a type, that can be represented by integer or
|
||||
# String or FixedString (possibly LowCardinality) or UUID or IPv6"
|
||||
# - It's not a composite type, e.g. `Nullable(Array(...))` is not a valid type
|
||||
parent = expression.parent
|
||||
if (
|
||||
expression.args.get("nullable") is not False
|
||||
and not (
|
||||
isinstance(parent, exp.DataType)
|
||||
and parent.is_type(exp.DataType.Type.MAP, check_nullable=True)
|
||||
and expression.index in (None, 0)
|
||||
)
|
||||
and not expression.is_type(*self.NON_NULLABLE_TYPES, check_nullable=True)
|
||||
):
|
||||
dtype = f"Nullable({dtype})"
|
||||
|
||||
return dtype
|
||||
|
||||
def cte_sql(self, expression: exp.CTE) -> str:
|
||||
if expression.args.get("scalar"):
|
||||
|
@ -953,7 +1044,10 @@ class ClickHouse(Dialect):
|
|||
if expression.kind in self.ON_CLUSTER_TARGETS and locations.get(
|
||||
exp.Properties.Location.POST_NAME
|
||||
):
|
||||
this_name = self.sql(expression.this, "this")
|
||||
this_name = self.sql(
|
||||
expression.this if isinstance(expression.this, exp.Schema) else expression,
|
||||
"this",
|
||||
)
|
||||
this_properties = " ".join(
|
||||
[self.sql(prop) for prop in locations[exp.Properties.Location.POST_NAME]]
|
||||
)
|
||||
|
@ -962,6 +1056,24 @@ class ClickHouse(Dialect):
|
|||
|
||||
return super().createable_sql(expression, locations)
|
||||
|
||||
def create_sql(self, expression: exp.Create) -> str:
|
||||
# The comment property comes last in CTAS statements, i.e. after the query
|
||||
query = expression.expression
|
||||
if isinstance(query, exp.Query):
|
||||
comment_prop = expression.find(exp.SchemaCommentProperty)
|
||||
if comment_prop:
|
||||
comment_prop.pop()
|
||||
query.replace(exp.paren(query))
|
||||
else:
|
||||
comment_prop = None
|
||||
|
||||
create_sql = super().create_sql(expression)
|
||||
|
||||
comment_sql = self.sql(comment_prop)
|
||||
comment_sql = f" {comment_sql}" if comment_sql else ""
|
||||
|
||||
return f"{create_sql}{comment_sql}"
|
||||
|
||||
def prewhere_sql(self, expression: exp.PreWhere) -> str:
|
||||
this = self.indent(self.sql(expression, "this"))
|
||||
return f"{self.seg('PREWHERE')}{self.sep()}{this}"
|
||||
|
|
|
@ -133,6 +133,10 @@ class _Dialect(type):
|
|||
klass.INVERSE_FORMAT_MAPPING = {v: k for k, v in klass.FORMAT_MAPPING.items()}
|
||||
klass.INVERSE_FORMAT_TRIE = new_trie(klass.INVERSE_FORMAT_MAPPING)
|
||||
|
||||
klass.INVERSE_CREATABLE_KIND_MAPPING = {
|
||||
v: k for k, v in klass.CREATABLE_KIND_MAPPING.items()
|
||||
}
|
||||
|
||||
base = seq_get(bases, 0)
|
||||
base_tokenizer = (getattr(base, "tokenizer_class", Tokenizer),)
|
||||
base_jsonpath_tokenizer = (getattr(base, "jsonpath_tokenizer_class", JSONPathTokenizer),)
|
||||
|
@ -183,6 +187,9 @@ class _Dialect(type):
|
|||
if enum not in ("", "bigquery"):
|
||||
klass.generator_class.SELECT_KINDS = ()
|
||||
|
||||
if enum not in ("", "clickhouse"):
|
||||
klass.generator_class.SUPPORTS_NULLABLE_TYPES = False
|
||||
|
||||
if enum not in ("", "athena", "presto", "trino"):
|
||||
klass.generator_class.TRY_SUPPORTED = False
|
||||
klass.generator_class.SUPPORTS_UESCAPE = False
|
||||
|
@ -369,6 +376,24 @@ class Dialect(metaclass=_Dialect):
|
|||
Whether ORDER BY ALL is supported (expands to all the selected columns) as in DuckDB, Spark3/Databricks
|
||||
"""
|
||||
|
||||
HAS_DISTINCT_ARRAY_CONSTRUCTORS = False
|
||||
"""
|
||||
Whether the ARRAY constructor is context-sensitive, i.e in Redshift ARRAY[1, 2, 3] != ARRAY(1, 2, 3)
|
||||
as the former is of type INT[] vs the latter which is SUPER
|
||||
"""
|
||||
|
||||
SUPPORTS_FIXED_SIZE_ARRAYS = False
|
||||
"""
|
||||
Whether expressions such as x::INT[5] should be parsed as fixed-size array defs/casts e.g. in DuckDB. In
|
||||
dialects which don't support fixed size arrays such as Snowflake, this should be interpreted as a subscript/index operator
|
||||
"""
|
||||
|
||||
CREATABLE_KIND_MAPPING: dict[str, str] = {}
|
||||
"""
|
||||
Helper for dialects that use a different name for the same creatable kind. For example, the Clickhouse
|
||||
equivalent of CREATE SCHEMA is CREATE DATABASE.
|
||||
"""
|
||||
|
||||
# --- Autofilled ---
|
||||
|
||||
tokenizer_class = Tokenizer
|
||||
|
@ -385,6 +410,8 @@ class Dialect(metaclass=_Dialect):
|
|||
INVERSE_FORMAT_MAPPING: t.Dict[str, str] = {}
|
||||
INVERSE_FORMAT_TRIE: t.Dict = {}
|
||||
|
||||
INVERSE_CREATABLE_KIND_MAPPING: dict[str, str] = {}
|
||||
|
||||
ESCAPED_SEQUENCES: t.Dict[str, str] = {}
|
||||
|
||||
# Delimiters for string literals and identifiers
|
||||
|
@ -635,6 +662,9 @@ class Dialect(metaclass=_Dialect):
|
|||
exp.GenerateDateArray: lambda self, e: self._annotate_with_type(
|
||||
e, exp.DataType.build("ARRAY<DATE>")
|
||||
),
|
||||
exp.GenerateTimestampArray: lambda self, e: self._annotate_with_type(
|
||||
e, exp.DataType.build("ARRAY<TIMESTAMP>")
|
||||
),
|
||||
exp.If: lambda self, e: self._annotate_by_args(e, "true", "false"),
|
||||
exp.Interval: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.INTERVAL),
|
||||
exp.Least: lambda self, e: self._annotate_by_args(e, "expressions"),
|
||||
|
@ -1214,7 +1244,13 @@ def right_to_substring_sql(self: Generator, expression: exp.Left) -> str:
|
|||
|
||||
|
||||
def timestrtotime_sql(self: Generator, expression: exp.TimeStrToTime) -> str:
|
||||
return self.sql(exp.cast(expression.this, exp.DataType.Type.TIMESTAMP))
|
||||
datatype = (
|
||||
exp.DataType.Type.TIMESTAMPTZ
|
||||
if expression.args.get("zone")
|
||||
else exp.DataType.Type.TIMESTAMP
|
||||
)
|
||||
|
||||
return self.sql(exp.cast(expression.this, datatype, dialect=self.dialect))
|
||||
|
||||
|
||||
def datestrtodate_sql(self: Generator, expression: exp.DateStrToDate) -> str:
|
||||
|
@ -1464,7 +1500,12 @@ def merge_without_target_sql(self: Generator, expression: exp.Merge) -> str:
|
|||
targets.add(normalize(alias.this))
|
||||
|
||||
for when in expression.expressions:
|
||||
when.transform(
|
||||
# only remove the target names from the THEN clause
|
||||
# theyre still valid in the <condition> part of WHEN MATCHED / WHEN NOT MATCHED
|
||||
# ref: https://github.com/TobikoData/sqlmesh/issues/2934
|
||||
then = when.args.get("then")
|
||||
if then:
|
||||
then.transform(
|
||||
lambda node: (
|
||||
exp.column(node.this)
|
||||
if isinstance(node, exp.Column) and normalize(node.args.get("table")) in targets
|
||||
|
@ -1590,9 +1631,9 @@ def sha256_sql(self: Generator, expression: exp.SHA2) -> str:
|
|||
return self.func(f"SHA{expression.text('length') or '256'}", expression.this)
|
||||
|
||||
|
||||
def sequence_sql(self: Generator, expression: exp.GenerateSeries):
|
||||
start = expression.args["start"]
|
||||
end = expression.args["end"]
|
||||
def sequence_sql(self: Generator, expression: exp.GenerateSeries | exp.GenerateDateArray) -> str:
|
||||
start = expression.args.get("start")
|
||||
end = expression.args.get("end")
|
||||
step = expression.args.get("step")
|
||||
|
||||
if isinstance(start, exp.Cast):
|
||||
|
@ -1602,8 +1643,8 @@ def sequence_sql(self: Generator, expression: exp.GenerateSeries):
|
|||
else:
|
||||
target_type = None
|
||||
|
||||
if target_type and target_type.is_type("timestamp"):
|
||||
if target_type is start.to:
|
||||
if start and end and target_type and target_type.is_type("date", "timestamp"):
|
||||
if isinstance(start, exp.Cast) and target_type is start.to:
|
||||
end = exp.cast(end, target_type)
|
||||
else:
|
||||
start = exp.cast(start, target_type)
|
||||
|
|
|
@ -32,6 +32,7 @@ from sqlglot.dialects.dialect import (
|
|||
timestamptrunc_sql,
|
||||
timestrtotime_sql,
|
||||
unit_to_var,
|
||||
unit_to_str,
|
||||
)
|
||||
from sqlglot.helper import seq_get
|
||||
from sqlglot.tokens import TokenType
|
||||
|
@ -81,6 +82,16 @@ def _date_sql(self: DuckDB.Generator, expression: exp.Date) -> str:
|
|||
return result
|
||||
|
||||
|
||||
# BigQuery -> DuckDB conversion for the TIME_DIFF function
|
||||
def _timediff_sql(self: DuckDB.Generator, expression: exp.TimeDiff) -> str:
|
||||
this = exp.cast(expression.this, exp.DataType.Type.TIME)
|
||||
expr = exp.cast(expression.expression, exp.DataType.Type.TIME)
|
||||
|
||||
# Although the 2 dialects share similar signatures, BQ seems to inverse
|
||||
# the sign of the result so the start/end time operands are flipped
|
||||
return self.func("DATE_DIFF", unit_to_str(expression), expr, this)
|
||||
|
||||
|
||||
def _array_sort_sql(self: DuckDB.Generator, expression: exp.ArraySort) -> str:
|
||||
if expression.expression:
|
||||
self.unsupported("DuckDB ARRAY_SORT does not support a comparator")
|
||||
|
@ -160,8 +171,10 @@ def _datatype_sql(self: DuckDB.Generator, expression: exp.DataType) -> str:
|
|||
if expression.is_type("array"):
|
||||
return f"{self.expressions(expression, flat=True)}[{self.expressions(expression, key='values', flat=True)}]"
|
||||
|
||||
# Type TIMESTAMP / TIME WITH TIME ZONE does not support any modifiers
|
||||
if expression.is_type("timestamptz", "timetz"):
|
||||
# Modifiers are not supported for TIME, [TIME | TIMESTAMP] WITH TIME ZONE
|
||||
if expression.is_type(
|
||||
exp.DataType.Type.TIME, exp.DataType.Type.TIMETZ, exp.DataType.Type.TIMESTAMPTZ
|
||||
):
|
||||
return expression.this.value
|
||||
|
||||
return self.datatype_sql(expression)
|
||||
|
@ -198,6 +211,41 @@ def _arrow_json_extract_sql(self: DuckDB.Generator, expression: JSON_EXTRACT_TYP
|
|||
return arrow_sql
|
||||
|
||||
|
||||
def _implicit_datetime_cast(
|
||||
arg: t.Optional[exp.Expression], type: exp.DataType.Type = exp.DataType.Type.DATE
|
||||
) -> t.Optional[exp.Expression]:
|
||||
return exp.cast(arg, type) if isinstance(arg, exp.Literal) else arg
|
||||
|
||||
|
||||
def _date_diff_sql(self: DuckDB.Generator, expression: exp.DateDiff) -> str:
|
||||
this = _implicit_datetime_cast(expression.this)
|
||||
expr = _implicit_datetime_cast(expression.expression)
|
||||
|
||||
return self.func("DATE_DIFF", unit_to_str(expression), expr, this)
|
||||
|
||||
|
||||
def _generate_datetime_array_sql(
|
||||
self: DuckDB.Generator, expression: t.Union[exp.GenerateDateArray, exp.GenerateTimestampArray]
|
||||
) -> str:
|
||||
is_generate_date_array = isinstance(expression, exp.GenerateDateArray)
|
||||
|
||||
type = exp.DataType.Type.DATE if is_generate_date_array else exp.DataType.Type.TIMESTAMP
|
||||
start = _implicit_datetime_cast(expression.args.get("start"), type=type)
|
||||
end = _implicit_datetime_cast(expression.args.get("end"), type=type)
|
||||
|
||||
# BQ's GENERATE_DATE_ARRAY & GENERATE_TIMESTAMP_ARRAY are transformed to DuckDB'S GENERATE_SERIES
|
||||
gen_series: t.Union[exp.GenerateSeries, exp.Cast] = exp.GenerateSeries(
|
||||
start=start, end=end, step=expression.args.get("step")
|
||||
)
|
||||
|
||||
if is_generate_date_array:
|
||||
# The GENERATE_SERIES result type is TIMESTAMP array, so to match BQ's semantics for
|
||||
# GENERATE_DATE_ARRAY we must cast it back to DATE array
|
||||
gen_series = exp.cast(gen_series, exp.DataType.build("ARRAY<DATE>"))
|
||||
|
||||
return self.sql(gen_series)
|
||||
|
||||
|
||||
class DuckDB(Dialect):
|
||||
NULL_ORDERING = "nulls_are_last"
|
||||
SUPPORTS_USER_DEFINED_TYPES = False
|
||||
|
@ -205,6 +253,7 @@ class DuckDB(Dialect):
|
|||
INDEX_OFFSET = 1
|
||||
CONCAT_COALESCE = True
|
||||
SUPPORTS_ORDER_BY_ALL = True
|
||||
SUPPORTS_FIXED_SIZE_ARRAYS = True
|
||||
|
||||
# https://duckdb.org/docs/sql/introduction.html#creating-a-new-table
|
||||
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
|
||||
|
@ -293,7 +342,7 @@ class DuckDB(Dialect):
|
|||
"LIST_HAS": exp.ArrayContains.from_arg_list,
|
||||
"LIST_REVERSE_SORT": _build_sort_array_desc,
|
||||
"LIST_SORT": exp.SortArray.from_arg_list,
|
||||
"LIST_VALUE": exp.Array.from_arg_list,
|
||||
"LIST_VALUE": lambda args: exp.Array(expressions=args),
|
||||
"MAKE_TIME": exp.TimeFromParts.from_arg_list,
|
||||
"MAKE_TIMESTAMP": _build_make_timestamp,
|
||||
"MEDIAN": lambda args: exp.PercentileCont(
|
||||
|
@ -416,6 +465,7 @@ class DuckDB(Dialect):
|
|||
COPY_HAS_INTO_KEYWORD = False
|
||||
STAR_EXCEPT = "EXCLUDE"
|
||||
PAD_FILL_PATTERN_IS_REQUIRED = True
|
||||
ARRAY_CONCAT_IS_VAR_LEN = False
|
||||
|
||||
TRANSFORMS = {
|
||||
**generator.Generator.TRANSFORMS,
|
||||
|
@ -441,9 +491,7 @@ class DuckDB(Dialect):
|
|||
exp.DateAdd: _date_delta_sql,
|
||||
exp.DateFromParts: rename_func("MAKE_DATE"),
|
||||
exp.DateSub: _date_delta_sql,
|
||||
exp.DateDiff: lambda self, e: self.func(
|
||||
"DATE_DIFF", f"'{e.args.get('unit') or 'DAY'}'", e.expression, e.this
|
||||
),
|
||||
exp.DateDiff: _date_diff_sql,
|
||||
exp.DateStrToDate: datestrtodate_sql,
|
||||
exp.Datetime: no_datetime_sql,
|
||||
exp.DatetimeSub: _date_delta_sql,
|
||||
|
@ -454,6 +502,8 @@ class DuckDB(Dialect):
|
|||
exp.DiToDate: lambda self,
|
||||
e: f"CAST(STRPTIME(CAST({self.sql(e, 'this')} AS TEXT), {DuckDB.DATEINT_FORMAT}) AS DATE)",
|
||||
exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False),
|
||||
exp.GenerateDateArray: _generate_datetime_array_sql,
|
||||
exp.GenerateTimestampArray: _generate_datetime_array_sql,
|
||||
exp.Explode: rename_func("UNNEST"),
|
||||
exp.IntDiv: lambda self, e: self.binary(e, "//"),
|
||||
exp.IsInf: rename_func("ISINF"),
|
||||
|
@ -498,6 +548,7 @@ class DuckDB(Dialect):
|
|||
exp.Struct: _struct_sql,
|
||||
exp.TimeAdd: _date_delta_sql,
|
||||
exp.Time: no_time_sql,
|
||||
exp.TimeDiff: _timediff_sql,
|
||||
exp.Timestamp: no_timestamp_sql,
|
||||
exp.TimestampDiff: lambda self, e: self.func(
|
||||
"DATE_DIFF", exp.Literal.string(e.unit), e.expression, e.this
|
||||
|
@ -522,6 +573,9 @@ class DuckDB(Dialect):
|
|||
exp.UnixToStr: lambda self, e: self.func(
|
||||
"STRFTIME", self.func("TO_TIMESTAMP", e.this), self.format_time(e)
|
||||
),
|
||||
exp.DatetimeTrunc: lambda self, e: self.func(
|
||||
"DATE_TRUNC", unit_to_str(e), exp.cast(e.this, exp.DataType.Type.DATETIME)
|
||||
),
|
||||
exp.UnixToTime: _unix_to_time_sql,
|
||||
exp.UnixToTimeStr: lambda self, e: f"CAST(TO_TIMESTAMP({self.sql(e, 'this')}) AS TEXT)",
|
||||
exp.VariancePop: rename_func("VAR_POP"),
|
||||
|
@ -650,6 +704,9 @@ class DuckDB(Dialect):
|
|||
PROPERTIES_LOCATION[exp.TemporaryProperty] = exp.Properties.Location.POST_CREATE
|
||||
PROPERTIES_LOCATION[exp.ReturnsProperty] = exp.Properties.Location.POST_ALIAS
|
||||
|
||||
def fromiso8601timestamp_sql(self, expression: exp.FromISO8601Timestamp) -> str:
|
||||
return self.sql(exp.cast(expression.this, exp.DataType.Type.TIMESTAMPTZ))
|
||||
|
||||
def strtotime_sql(self, expression: exp.StrToTime) -> str:
|
||||
if expression.args.get("safe"):
|
||||
formatted_time = self.format_time(expression)
|
||||
|
@ -832,3 +889,24 @@ class DuckDB(Dialect):
|
|||
return self.func("STRUCT_PACK", kv_sql)
|
||||
|
||||
return self.func("STRUCT_INSERT", this, kv_sql)
|
||||
|
||||
def unnest_sql(self, expression: exp.Unnest) -> str:
|
||||
explode_array = expression.args.get("explode_array")
|
||||
if explode_array:
|
||||
# In BigQuery, UNNESTing a nested array leads to explosion of the top-level array & struct
|
||||
# This is transpiled to DDB by transforming "FROM UNNEST(...)" to "FROM (SELECT UNNEST(..., max_depth => 2))"
|
||||
expression.expressions.append(
|
||||
exp.Kwarg(this=exp.var("max_depth"), expression=exp.Literal.number(2))
|
||||
)
|
||||
|
||||
# If BQ's UNNEST is aliased, we transform it from a column alias to a table alias in DDB
|
||||
alias = expression.args.get("alias")
|
||||
if alias:
|
||||
expression.set("alias", None)
|
||||
alias = exp.TableAlias(this=seq_get(alias.args.get("columns"), 0))
|
||||
|
||||
unnest_sql = super().unnest_sql(expression)
|
||||
select = exp.Select(expressions=[unnest_sql]).subquery(alias)
|
||||
return self.sql(select)
|
||||
|
||||
return super().unnest_sql(expression)
|
||||
|
|
|
@ -252,6 +252,7 @@ class Hive(Dialect):
|
|||
"ADD FILES": TokenType.COMMAND,
|
||||
"ADD JAR": TokenType.COMMAND,
|
||||
"ADD JARS": TokenType.COMMAND,
|
||||
"MINUS": TokenType.EXCEPT,
|
||||
"MSCK REPAIR": TokenType.COMMAND,
|
||||
"REFRESH": TokenType.REFRESH,
|
||||
"TIMESTAMP AS OF": TokenType.TIMESTAMP_SNAPSHOT,
|
||||
|
@ -509,6 +510,7 @@ class Hive(Dialect):
|
|||
e: f"STORED AS {self.sql(e, 'this') if isinstance(e.this, exp.InputOutputFormat) else e.name.upper()}",
|
||||
exp.FromBase64: rename_func("UNBASE64"),
|
||||
exp.GenerateSeries: sequence_sql,
|
||||
exp.GenerateDateArray: sequence_sql,
|
||||
exp.If: if_sql(),
|
||||
exp.ILike: no_ilike_sql,
|
||||
exp.IsNan: rename_func("ISNAN"),
|
||||
|
@ -552,6 +554,7 @@ class Hive(Dialect):
|
|||
exp.StrToTime: _str_to_time_sql,
|
||||
exp.StrToUnix: _str_to_unix_sql,
|
||||
exp.StructExtract: struct_extract_sql,
|
||||
exp.Table: transforms.preprocess([transforms.unnest_generate_series]),
|
||||
exp.TimeStrToDate: rename_func("TO_DATE"),
|
||||
exp.TimeStrToTime: timestrtotime_sql,
|
||||
exp.TimeStrToUnix: rename_func("UNIX_TIMESTAMP"),
|
||||
|
@ -570,6 +573,7 @@ class Hive(Dialect):
|
|||
),
|
||||
exp.UnixToTime: _unix_to_time_sql,
|
||||
exp.UnixToTimeStr: rename_func("FROM_UNIXTIME"),
|
||||
exp.Unnest: rename_func("EXPLODE"),
|
||||
exp.PartitionedByProperty: lambda self, e: f"PARTITIONED BY {self.sql(e, 'this')}",
|
||||
exp.NumberToStr: rename_func("FORMAT_NUMBER"),
|
||||
exp.National: lambda self, e: self.national_sql(e, prefix=""),
|
||||
|
@ -593,6 +597,9 @@ class Hive(Dialect):
|
|||
exp.WithDataProperty: exp.Properties.Location.UNSUPPORTED,
|
||||
}
|
||||
|
||||
def unnest_sql(self, expression: exp.Unnest) -> str:
|
||||
return rename_func("EXPLODE")(self, expression)
|
||||
|
||||
def _jsonpathkey_sql(self, expression: exp.JSONPathKey) -> str:
|
||||
if isinstance(expression.this, exp.JSONPathWildcard):
|
||||
self.unsupported("Unsupported wildcard in JSONPathKey expression")
|
||||
|
|
|
@ -302,6 +302,9 @@ class MySQL(Dialect):
|
|||
|
||||
FUNCTIONS = {
|
||||
**parser.Parser.FUNCTIONS,
|
||||
"CONVERT_TZ": lambda args: exp.ConvertTimezone(
|
||||
source_tz=seq_get(args, 1), target_tz=seq_get(args, 2), timestamp=seq_get(args, 0)
|
||||
),
|
||||
"DATE": lambda args: exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
"DATE_ADD": build_date_delta_with_interval(exp.DateAdd),
|
||||
"DATE_FORMAT": build_formatted_time(exp.TimeToStr, "mysql"),
|
||||
|
@ -724,6 +727,7 @@ class MySQL(Dialect):
|
|||
transforms.eliminate_semi_and_anti_joins,
|
||||
transforms.eliminate_qualify,
|
||||
transforms.eliminate_full_outer_join,
|
||||
transforms.unnest_generate_date_array_using_recursive_cte,
|
||||
]
|
||||
),
|
||||
exp.StrPosition: strposition_to_locate_sql,
|
||||
|
@ -1213,3 +1217,10 @@ class MySQL(Dialect):
|
|||
dateadd = build_date_delta_with_interval(exp.DateAdd)([start_ts, interval])
|
||||
|
||||
return self.sql(dateadd)
|
||||
|
||||
def converttimezone_sql(self, expression: exp.ConvertTimezone) -> str:
|
||||
from_tz = expression.args.get("source_tz")
|
||||
to_tz = expression.args.get("target_tz")
|
||||
dt = expression.args.get("timestamp")
|
||||
|
||||
return self.func("CONVERT_TZ", dt, from_tz, to_tz)
|
||||
|
|
|
@ -78,6 +78,8 @@ class Oracle(Dialect):
|
|||
for prefix in ("U", "u")
|
||||
]
|
||||
|
||||
NESTED_COMMENTS = False
|
||||
|
||||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
"(+)": TokenType.JOIN_MARKER,
|
||||
|
@ -90,7 +92,6 @@ class Oracle(Dialect):
|
|||
"ORDER SIBLINGS BY": TokenType.ORDER_SIBLINGS_BY,
|
||||
"SAMPLE": TokenType.TABLE_SAMPLE,
|
||||
"START": TokenType.BEGIN,
|
||||
"SYSDATE": TokenType.CURRENT_TIMESTAMP,
|
||||
"TOP": TokenType.TOP,
|
||||
"VARCHAR2": TokenType.VARCHAR,
|
||||
}
|
||||
|
@ -106,8 +107,15 @@ class Oracle(Dialect):
|
|||
"TO_CHAR": _build_timetostr_or_tochar,
|
||||
"TO_TIMESTAMP": build_formatted_time(exp.StrToTime, "oracle"),
|
||||
"TO_DATE": build_formatted_time(exp.StrToDate, "oracle"),
|
||||
"NVL": lambda args: exp.Coalesce(
|
||||
this=seq_get(args, 0), expressions=args[1:], is_nvl=True
|
||||
),
|
||||
}
|
||||
|
||||
NO_PAREN_FUNCTION_PARSERS = {
|
||||
**parser.Parser.NO_PAREN_FUNCTION_PARSERS,
|
||||
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, sysdate=True),
|
||||
}
|
||||
FUNCTIONS.pop("NVL")
|
||||
|
||||
FUNCTION_PARSERS: t.Dict[str, t.Callable] = {
|
||||
**parser.Parser.FUNCTION_PARSERS,
|
||||
|
@ -247,6 +255,7 @@ class Oracle(Dialect):
|
|||
),
|
||||
exp.Group: transforms.preprocess([transforms.unalias_group]),
|
||||
exp.ILike: no_ilike_sql,
|
||||
exp.Mod: rename_func("MOD"),
|
||||
exp.Select: transforms.preprocess(
|
||||
[
|
||||
transforms.eliminate_distinct_on,
|
||||
|
@ -274,6 +283,9 @@ class Oracle(Dialect):
|
|||
}
|
||||
|
||||
def currenttimestamp_sql(self, expression: exp.CurrentTimestamp) -> str:
|
||||
if expression.args.get("sysdate"):
|
||||
return "SYSDATE"
|
||||
|
||||
this = expression.this
|
||||
return self.func("CURRENT_TIMESTAMP", this) if this else "CURRENT_TIMESTAMP"
|
||||
|
||||
|
@ -291,7 +303,7 @@ class Oracle(Dialect):
|
|||
)
|
||||
return f"XMLTABLE({self.sep('')}{self.indent(this + passing + by_ref + columns)}{self.seg(')', sep='')}"
|
||||
|
||||
def add_column_sql(self, expression: exp.AlterTable) -> str:
|
||||
def add_column_sql(self, expression: exp.Alter) -> str:
|
||||
actions = self.expressions(expression, key="actions", flat=True)
|
||||
if len(expression.args.get("actions", [])) > 1:
|
||||
return f"ADD ({actions})"
|
||||
|
@ -303,3 +315,7 @@ class Oracle(Dialect):
|
|||
value = f" CONSTRAINT {value}" if value else ""
|
||||
|
||||
return f"{option}{value}"
|
||||
|
||||
def coalesce_sql(self, expression: exp.Coalesce) -> str:
|
||||
func_name = "NVL" if expression.args.get("is_nvl") else "COALESCE"
|
||||
return rename_func(func_name)(self, expression)
|
||||
|
|
|
@ -166,7 +166,7 @@ def _serial_to_generated(expression: exp.Expression) -> exp.Expression:
|
|||
return expression
|
||||
|
||||
|
||||
def _build_generate_series(args: t.List) -> exp.GenerateSeries:
|
||||
def _build_generate_series(args: t.List) -> exp.ExplodingGenerateSeries:
|
||||
# The goal is to convert step values like '1 day' or INTERVAL '1 day' into INTERVAL '1' day
|
||||
# Note: postgres allows calls with just two arguments -- the "step" argument defaults to 1
|
||||
step = seq_get(args, 2)
|
||||
|
@ -176,7 +176,7 @@ def _build_generate_series(args: t.List) -> exp.GenerateSeries:
|
|||
elif isinstance(step, exp.Interval) and not step.args.get("unit"):
|
||||
args[2] = exp.to_interval(step.this.this)
|
||||
|
||||
return exp.GenerateSeries.from_arg_list(args)
|
||||
return exp.ExplodingGenerateSeries.from_arg_list(args)
|
||||
|
||||
|
||||
def _build_to_timestamp(args: t.List) -> exp.UnixToTime | exp.StrToTime:
|
||||
|
@ -440,7 +440,7 @@ class Postgres(Dialect):
|
|||
self._match(TokenType.COMMA)
|
||||
value = self._parse_bitwise()
|
||||
|
||||
if part and part.is_string:
|
||||
if part and isinstance(part, (exp.Column, exp.Literal)):
|
||||
part = exp.var(part.name)
|
||||
|
||||
return self.expression(exp.Extract, this=part, expression=value)
|
||||
|
@ -466,6 +466,7 @@ class Postgres(Dialect):
|
|||
MULTI_ARG_DISTINCT = False
|
||||
CAN_IMPLEMENT_ARRAY_ANY = True
|
||||
COPY_HAS_INTO_KEYWORD = False
|
||||
ARRAY_CONCAT_IS_VAR_LEN = False
|
||||
|
||||
SUPPORTED_JSON_PATH_PARTS = {
|
||||
exp.JSONPathKey,
|
||||
|
@ -487,12 +488,7 @@ class Postgres(Dialect):
|
|||
TRANSFORMS = {
|
||||
**generator.Generator.TRANSFORMS,
|
||||
exp.AnyValue: any_value_to_max_sql,
|
||||
exp.Array: lambda self, e: (
|
||||
f"{self.normalize_func('ARRAY')}({self.sql(e.expressions[0])})"
|
||||
if isinstance(seq_get(e.expressions, 0), exp.Select)
|
||||
else f"{self.normalize_func('ARRAY')}[{self.expressions(e, flat=True)}]"
|
||||
),
|
||||
exp.ArrayConcat: rename_func("ARRAY_CAT"),
|
||||
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CAT"),
|
||||
exp.ArrayContainsAll: lambda self, e: self.binary(e, "@>"),
|
||||
exp.ArrayOverlaps: lambda self, e: self.binary(e, "&&"),
|
||||
exp.ArrayFilter: filter_array_using_unnest,
|
||||
|
@ -507,6 +503,7 @@ class Postgres(Dialect):
|
|||
exp.DateStrToDate: datestrtodate_sql,
|
||||
exp.DateSub: _date_add_sql("-"),
|
||||
exp.Explode: rename_func("UNNEST"),
|
||||
exp.ExplodingGenerateSeries: rename_func("GENERATE_SERIES"),
|
||||
exp.GroupConcat: _string_agg_sql,
|
||||
exp.IntDiv: rename_func("DIV"),
|
||||
exp.JSONExtract: _json_extract_sql("JSON_EXTRACT_PATH", "->"),
|
||||
|
@ -587,21 +584,32 @@ class Postgres(Dialect):
|
|||
|
||||
def unnest_sql(self, expression: exp.Unnest) -> str:
|
||||
if len(expression.expressions) == 1:
|
||||
arg = expression.expressions[0]
|
||||
if isinstance(arg, exp.GenerateDateArray):
|
||||
generate_series: exp.Expression = exp.GenerateSeries(**arg.args)
|
||||
if isinstance(expression.parent, (exp.From, exp.Join)):
|
||||
generate_series = (
|
||||
exp.select("value::date")
|
||||
.from_(generate_series.as_("value"))
|
||||
.subquery(expression.args.get("alias") or "_unnested_generate_series")
|
||||
)
|
||||
return self.sql(generate_series)
|
||||
|
||||
from sqlglot.optimizer.annotate_types import annotate_types
|
||||
|
||||
this = annotate_types(expression.expressions[0])
|
||||
this = annotate_types(arg)
|
||||
if this.is_type("array<json>"):
|
||||
while isinstance(this, exp.Cast):
|
||||
this = this.this
|
||||
|
||||
arg = self.sql(exp.cast(this, exp.DataType.Type.JSON))
|
||||
arg_as_json = self.sql(exp.cast(this, exp.DataType.Type.JSON))
|
||||
alias = self.sql(expression, "alias")
|
||||
alias = f" AS {alias}" if alias else ""
|
||||
|
||||
if expression.args.get("offset"):
|
||||
self.unsupported("Unsupported JSON_ARRAY_ELEMENTS with offset")
|
||||
|
||||
return f"JSON_ARRAY_ELEMENTS({arg}){alias}"
|
||||
return f"JSON_ARRAY_ELEMENTS({arg_as_json}){alias}"
|
||||
|
||||
return super().unnest_sql(expression)
|
||||
|
||||
|
@ -646,3 +654,11 @@ class Postgres(Dialect):
|
|||
return self.sql(this)
|
||||
|
||||
return super().cast_sql(expression, safe_prefix=safe_prefix)
|
||||
|
||||
def array_sql(self, expression: exp.Array) -> str:
|
||||
exprs = expression.expressions
|
||||
return (
|
||||
f"{self.normalize_func('ARRAY')}({self.sql(exprs[0])})"
|
||||
if isinstance(seq_get(exprs, 0), exp.Select)
|
||||
else f"{self.normalize_func('ARRAY')}[{self.expressions(expression, flat=True)}]"
|
||||
)
|
||||
|
|
|
@ -142,17 +142,6 @@ def _build_from_unixtime(args: t.List) -> exp.Expression:
|
|||
return exp.UnixToTime.from_arg_list(args)
|
||||
|
||||
|
||||
def _unnest_sequence(expression: exp.Expression) -> exp.Expression:
|
||||
if isinstance(expression, exp.Table):
|
||||
if isinstance(expression.this, exp.GenerateSeries):
|
||||
unnest = exp.Unnest(expressions=[expression.this])
|
||||
|
||||
if expression.alias:
|
||||
return exp.alias_(unnest, alias="_u", table=[expression.alias], copy=False)
|
||||
return unnest
|
||||
return expression
|
||||
|
||||
|
||||
def _first_last_sql(self: Presto.Generator, expression: exp.Func) -> str:
|
||||
"""
|
||||
Trino doesn't support FIRST / LAST as functions, but they're valid in the context
|
||||
|
@ -245,13 +234,17 @@ class Presto(Dialect):
|
|||
INDEX_OFFSET = 1
|
||||
NULL_ORDERING = "nulls_are_last"
|
||||
TIME_FORMAT = MySQL.TIME_FORMAT
|
||||
TIME_MAPPING = MySQL.TIME_MAPPING
|
||||
STRICT_STRING_CONCAT = True
|
||||
SUPPORTS_SEMI_ANTI_JOIN = False
|
||||
TYPED_DIVISION = True
|
||||
TABLESAMPLE_SIZE_IS_PERCENT = True
|
||||
LOG_BASE_FIRST: t.Optional[bool] = None
|
||||
|
||||
TIME_MAPPING = {
|
||||
**MySQL.TIME_MAPPING,
|
||||
"%W": "%A",
|
||||
}
|
||||
|
||||
# https://github.com/trinodb/trino/issues/17
|
||||
# https://github.com/trinodb/trino/issues/12289
|
||||
# https://github.com/prestodb/presto/issues/2863
|
||||
|
@ -434,6 +427,7 @@ class Presto(Dialect):
|
|||
exp.FromTimeZone: lambda self,
|
||||
e: f"WITH_TIMEZONE({self.sql(e, 'this')}, {self.sql(e, 'zone')}) AT TIME ZONE 'UTC'",
|
||||
exp.GenerateSeries: sequence_sql,
|
||||
exp.GenerateDateArray: sequence_sql,
|
||||
exp.Group: transforms.preprocess([transforms.unalias_group]),
|
||||
exp.GroupConcat: lambda self, e: self.func(
|
||||
"ARRAY_JOIN", self.func("ARRAY_AGG", e.this), e.args.get("separator")
|
||||
|
@ -471,7 +465,7 @@ class Presto(Dialect):
|
|||
exp.StrToMap: rename_func("SPLIT_TO_MAP"),
|
||||
exp.StrToTime: _str_to_time_sql,
|
||||
exp.StructExtract: struct_extract_sql,
|
||||
exp.Table: transforms.preprocess([_unnest_sequence]),
|
||||
exp.Table: transforms.preprocess([transforms.unnest_generate_series]),
|
||||
exp.Timestamp: no_timestamp_sql,
|
||||
exp.TimestampAdd: _date_delta_sql("DATE_ADD"),
|
||||
exp.TimestampTrunc: timestamptrunc_sql(),
|
||||
|
|
|
@ -17,6 +17,7 @@ from sqlglot.dialects.dialect import (
|
|||
from sqlglot.dialects.postgres import Postgres
|
||||
from sqlglot.helper import seq_get
|
||||
from sqlglot.tokens import TokenType
|
||||
from sqlglot.parser import build_convert_timezone
|
||||
|
||||
if t.TYPE_CHECKING:
|
||||
from sqlglot._typing import E
|
||||
|
@ -45,13 +46,11 @@ class Redshift(Postgres):
|
|||
INDEX_OFFSET = 0
|
||||
COPY_PARAMS_ARE_CSV = False
|
||||
HEX_LOWERCASE = True
|
||||
HAS_DISTINCT_ARRAY_CONSTRUCTORS = True
|
||||
|
||||
TIME_FORMAT = "'YYYY-MM-DD HH:MI:SS'"
|
||||
TIME_MAPPING = {
|
||||
**Postgres.TIME_MAPPING,
|
||||
"MON": "%b",
|
||||
"HH": "%H",
|
||||
}
|
||||
# ref: https://docs.aws.amazon.com/redshift/latest/dg/r_FORMAT_strings.html
|
||||
TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'"
|
||||
TIME_MAPPING = {**Postgres.TIME_MAPPING, "MON": "%b", "HH24": "%H", "HH": "%I"}
|
||||
|
||||
class Parser(Postgres.Parser):
|
||||
FUNCTIONS = {
|
||||
|
@ -62,6 +61,7 @@ class Redshift(Postgres):
|
|||
unit=exp.var("month"),
|
||||
return_type=exp.DataType.build("TIMESTAMP"),
|
||||
),
|
||||
"CONVERT_TIMEZONE": lambda args: build_convert_timezone(args, "UTC"),
|
||||
"DATEADD": _build_date_delta(exp.TsOrDsAdd),
|
||||
"DATE_ADD": _build_date_delta(exp.TsOrDsAdd),
|
||||
"DATEDIFF": _build_date_delta(exp.TsOrDsDiff),
|
||||
|
@ -77,7 +77,7 @@ class Redshift(Postgres):
|
|||
NO_PAREN_FUNCTION_PARSERS = {
|
||||
**Postgres.Parser.NO_PAREN_FUNCTION_PARSERS,
|
||||
"APPROXIMATE": lambda self: self._parse_approximate_count(),
|
||||
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, transaction=True),
|
||||
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, sysdate=True),
|
||||
}
|
||||
|
||||
SUPPORTS_IMPLICIT_UNNEST = True
|
||||
|
@ -153,6 +153,8 @@ class Redshift(Postgres):
|
|||
COPY_PARAMS_ARE_WRAPPED = False
|
||||
HEX_FUNC = "TO_HEX"
|
||||
PARSE_JSON_NAME = "JSON_PARSE"
|
||||
ARRAY_CONCAT_IS_VAR_LEN = False
|
||||
SUPPORTS_CONVERT_TIMEZONE = True
|
||||
|
||||
# Redshift doesn't have `WITH` as part of their with_properties so we remove it
|
||||
WITH_PROPERTIES_PREFIX = " "
|
||||
|
@ -169,12 +171,13 @@ class Redshift(Postgres):
|
|||
|
||||
TRANSFORMS = {
|
||||
**Postgres.Generator.TRANSFORMS,
|
||||
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CONCAT"),
|
||||
exp.Concat: concat_to_dpipe_sql,
|
||||
exp.ConcatWs: concat_ws_to_dpipe_sql,
|
||||
exp.ApproxDistinct: lambda self,
|
||||
e: f"APPROXIMATE COUNT(DISTINCT {self.sql(e, 'this')})",
|
||||
exp.CurrentTimestamp: lambda self, e: (
|
||||
"SYSDATE" if e.args.get("transaction") else "GETDATE()"
|
||||
"SYSDATE" if e.args.get("sysdate") else "GETDATE()"
|
||||
),
|
||||
exp.DateAdd: date_delta_sql("DATEADD"),
|
||||
exp.DateDiff: date_delta_sql("DATEDIFF"),
|
||||
|
@ -191,6 +194,7 @@ class Redshift(Postgres):
|
|||
transforms.eliminate_distinct_on,
|
||||
transforms.eliminate_semi_and_anti_joins,
|
||||
transforms.unqualify_unnest,
|
||||
transforms.unnest_generate_date_array_using_recursive_cte,
|
||||
]
|
||||
),
|
||||
exp.SortKeyProperty: lambda self,
|
||||
|
@ -423,3 +427,9 @@ class Redshift(Postgres):
|
|||
file_format = f" FILE FORMAT {file_format}" if file_format else ""
|
||||
|
||||
return f"SET{exprs}{location}{file_format}"
|
||||
|
||||
def array_sql(self, expression: exp.Array) -> str:
|
||||
if expression.args.get("bracket_notation"):
|
||||
return super().array_sql(expression)
|
||||
|
||||
return rename_func("ARRAY")(self, expression)
|
||||
|
|
|
@ -122,12 +122,6 @@ def _regexpilike_sql(self: Snowflake.Generator, expression: exp.RegexpILike) ->
|
|||
)
|
||||
|
||||
|
||||
def _build_convert_timezone(args: t.List) -> t.Union[exp.Anonymous, exp.AtTimeZone]:
|
||||
if len(args) == 3:
|
||||
return exp.Anonymous(this="CONVERT_TIMEZONE", expressions=args)
|
||||
return exp.AtTimeZone(this=seq_get(args, 1), zone=seq_get(args, 0))
|
||||
|
||||
|
||||
def _build_regexp_replace(args: t.List) -> exp.RegexpReplace:
|
||||
regexp_replace = exp.RegexpReplace.from_arg_list(args)
|
||||
|
||||
|
@ -186,6 +180,47 @@ def _flatten_structured_types_unless_iceberg(expression: exp.Expression) -> exp.
|
|||
return expression
|
||||
|
||||
|
||||
def _unnest_generate_date_array(expression: exp.Expression) -> exp.Expression:
|
||||
if isinstance(expression, exp.Select):
|
||||
for unnest in expression.find_all(exp.Unnest):
|
||||
if (
|
||||
isinstance(unnest.parent, (exp.From, exp.Join))
|
||||
and len(unnest.expressions) == 1
|
||||
and isinstance(unnest.expressions[0], exp.GenerateDateArray)
|
||||
):
|
||||
generate_date_array = unnest.expressions[0]
|
||||
start = generate_date_array.args.get("start")
|
||||
end = generate_date_array.args.get("end")
|
||||
step = generate_date_array.args.get("step")
|
||||
|
||||
if not start or not end or not isinstance(step, exp.Interval) or step.name != "1":
|
||||
continue
|
||||
|
||||
unit = step.args.get("unit")
|
||||
|
||||
unnest_alias = unnest.args.get("alias")
|
||||
if unnest_alias:
|
||||
unnest_alias = unnest_alias.copy()
|
||||
sequence_value_name = seq_get(unnest_alias.columns, 0) or "value"
|
||||
else:
|
||||
sequence_value_name = "value"
|
||||
|
||||
# We'll add the next sequence value to the starting date and project the result
|
||||
date_add = _build_date_time_add(exp.DateAdd)(
|
||||
[unit, exp.cast(sequence_value_name, "int"), exp.cast(start, "date")]
|
||||
).as_(sequence_value_name)
|
||||
|
||||
# We use DATEDIFF to compute the number of sequence values needed
|
||||
number_sequence = Snowflake.Parser.FUNCTIONS["ARRAY_GENERATE_RANGE"](
|
||||
[exp.Literal.number(0), _build_datediff([unit, start, end]) + 1]
|
||||
)
|
||||
|
||||
unnest.set("expressions", [number_sequence])
|
||||
unnest.replace(exp.select(date_add).from_(unnest.copy()).subquery(unnest_alias))
|
||||
|
||||
return expression
|
||||
|
||||
|
||||
class Snowflake(Dialect):
|
||||
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax
|
||||
NORMALIZATION_STRATEGY = NormalizationStrategy.UPPERCASE
|
||||
|
@ -255,7 +290,7 @@ class Snowflake(Dialect):
|
|||
**parser.Parser.FUNCTIONS,
|
||||
"APPROX_PERCENTILE": exp.ApproxQuantile.from_arg_list,
|
||||
"ARRAYAGG": exp.ArrayAgg.from_arg_list,
|
||||
"ARRAY_CONSTRUCT": exp.Array.from_arg_list,
|
||||
"ARRAY_CONSTRUCT": lambda args: exp.Array(expressions=args),
|
||||
"ARRAY_CONTAINS": lambda args: exp.ArrayContains(
|
||||
this=seq_get(args, 1), expression=seq_get(args, 0)
|
||||
),
|
||||
|
@ -268,7 +303,6 @@ class Snowflake(Dialect):
|
|||
"BITXOR": binary_from_function(exp.BitwiseXor),
|
||||
"BIT_XOR": binary_from_function(exp.BitwiseXor),
|
||||
"BOOLXOR": binary_from_function(exp.Xor),
|
||||
"CONVERT_TIMEZONE": _build_convert_timezone,
|
||||
"DATE": _build_datetime("DATE", exp.DataType.Type.DATE),
|
||||
"DATE_TRUNC": _date_trunc_to_time,
|
||||
"DATEADD": _build_date_time_add(exp.DateAdd),
|
||||
|
@ -413,6 +447,26 @@ class Snowflake(Dialect):
|
|||
),
|
||||
}
|
||||
|
||||
def _negate_range(
|
||||
self, this: t.Optional[exp.Expression] = None
|
||||
) -> t.Optional[exp.Expression]:
|
||||
if not this:
|
||||
return this
|
||||
|
||||
query = this.args.get("query")
|
||||
if isinstance(this, exp.In) and isinstance(query, exp.Query):
|
||||
# Snowflake treats `value NOT IN (subquery)` as `VALUE <> ALL (subquery)`, so
|
||||
# we do this conversion here to avoid parsing it into `NOT value IN (subquery)`
|
||||
# which can produce different results (most likely a SnowFlake bug).
|
||||
#
|
||||
# https://docs.snowflake.com/en/sql-reference/functions/in
|
||||
# Context: https://github.com/tobymao/sqlglot/issues/3890
|
||||
return self.expression(
|
||||
exp.NEQ, this=this.this, expression=exp.All(this=query.unnest())
|
||||
)
|
||||
|
||||
return self.expression(exp.Not, this=this)
|
||||
|
||||
def _parse_with_constraint(self) -> t.Optional[exp.Expression]:
|
||||
if self._prev.token_type != TokenType.WITH:
|
||||
self._retreat(self._index - 1)
|
||||
|
@ -638,6 +692,7 @@ class Snowflake(Dialect):
|
|||
HEX_STRINGS = [("x'", "'"), ("X'", "'")]
|
||||
RAW_STRINGS = ["$$"]
|
||||
COMMENTS = ["--", "//", ("/*", "*/")]
|
||||
NESTED_COMMENTS = False
|
||||
|
||||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
|
@ -692,6 +747,9 @@ class Snowflake(Dialect):
|
|||
COPY_PARAMS_ARE_WRAPPED = False
|
||||
COPY_PARAMS_EQ_REQUIRED = True
|
||||
STAR_EXCEPT = "EXCLUDE"
|
||||
SUPPORTS_EXPLODING_PROJECTIONS = False
|
||||
ARRAY_CONCAT_IS_VAR_LEN = False
|
||||
SUPPORTS_CONVERT_TIMEZONE = True
|
||||
|
||||
TRANSFORMS = {
|
||||
**generator.Generator.TRANSFORMS,
|
||||
|
@ -699,7 +757,7 @@ class Snowflake(Dialect):
|
|||
exp.ArgMax: rename_func("MAX_BY"),
|
||||
exp.ArgMin: rename_func("MIN_BY"),
|
||||
exp.Array: inline_array_sql,
|
||||
exp.ArrayConcat: rename_func("ARRAY_CAT"),
|
||||
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CAT"),
|
||||
exp.ArrayContains: lambda self, e: self.func("ARRAY_CONTAINS", e.expression, e.this),
|
||||
exp.AtTimeZone: lambda self, e: self.func(
|
||||
"CONVERT_TIMEZONE", e.args.get("zone"), e.this
|
||||
|
@ -751,6 +809,7 @@ class Snowflake(Dialect):
|
|||
transforms.eliminate_distinct_on,
|
||||
transforms.explode_to_unnest(),
|
||||
transforms.eliminate_semi_and_anti_joins,
|
||||
_unnest_generate_date_array,
|
||||
]
|
||||
),
|
||||
exp.SHA: rename_func("SHA1"),
|
||||
|
|
|
@ -132,6 +132,7 @@ class Spark(Spark2):
|
|||
class Generator(Spark2.Generator):
|
||||
SUPPORTS_TO_NUMBER = True
|
||||
PAD_FILL_PATTERN_IS_REQUIRED = False
|
||||
SUPPORTS_CONVERT_TIMEZONE = True
|
||||
|
||||
TYPE_MAPPING = {
|
||||
**Spark2.Generator.TYPE_MAPPING,
|
||||
|
|
|
@ -18,6 +18,7 @@ from sqlglot.dialects.dialect import (
|
|||
build_date_delta,
|
||||
rename_func,
|
||||
trim_sql,
|
||||
timestrtotime_sql,
|
||||
)
|
||||
from sqlglot.helper import seq_get
|
||||
from sqlglot.time import format_time
|
||||
|
@ -339,6 +340,16 @@ def _json_extract_sql(
|
|||
return self.func("ISNULL", json_query, json_value)
|
||||
|
||||
|
||||
def _timestrtotime_sql(self: TSQL.Generator, expression: exp.TimeStrToTime):
|
||||
sql = timestrtotime_sql(self, expression)
|
||||
if expression.args.get("zone"):
|
||||
# If there is a timezone, produce an expression like:
|
||||
# CAST('2020-01-01 12:13:14-08:00' AS DATETIMEOFFSET) AT TIME ZONE 'UTC'
|
||||
# If you dont have AT TIME ZONE 'UTC', wrapping that expression in another cast back to DATETIME2 just drops the timezone information
|
||||
return self.sql(exp.AtTimeZone(this=sql, zone=exp.Literal.string("UTC")))
|
||||
return sql
|
||||
|
||||
|
||||
class TSQL(Dialect):
|
||||
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
|
||||
TIME_FORMAT = "'yyyy-mm-dd hh:mm:ss'"
|
||||
|
@ -863,6 +874,7 @@ class TSQL(Dialect):
|
|||
transforms.eliminate_distinct_on,
|
||||
transforms.eliminate_semi_and_anti_joins,
|
||||
transforms.eliminate_qualify,
|
||||
transforms.unnest_generate_date_array_using_recursive_cte,
|
||||
]
|
||||
),
|
||||
exp.Stddev: rename_func("STDEV"),
|
||||
|
@ -875,9 +887,7 @@ class TSQL(Dialect):
|
|||
"HASHBYTES", exp.Literal.string(f"SHA2_{e.args.get('length', 256)}"), e.this
|
||||
),
|
||||
exp.TemporaryProperty: lambda self, e: "",
|
||||
exp.TimeStrToTime: lambda self, e: self.sql(
|
||||
exp.cast(e.this, exp.DataType.Type.DATETIME)
|
||||
),
|
||||
exp.TimeStrToTime: _timestrtotime_sql,
|
||||
exp.TimeToStr: _format_sql,
|
||||
exp.Trim: trim_sql,
|
||||
exp.TsOrDsAdd: date_delta_sql("DATEADD", cast=True),
|
||||
|
@ -1139,11 +1149,11 @@ class TSQL(Dialect):
|
|||
def partition_sql(self, expression: exp.Partition) -> str:
|
||||
return f"WITH (PARTITIONS({self.expressions(expression, flat=True)}))"
|
||||
|
||||
def altertable_sql(self, expression: exp.AlterTable) -> str:
|
||||
def alter_sql(self, expression: exp.Alter) -> str:
|
||||
action = seq_get(expression.args.get("actions") or [], 0)
|
||||
if isinstance(action, exp.RenameTable):
|
||||
return f"EXEC sp_rename '{self.sql(expression.this)}', '{action.this.name}'"
|
||||
return super().altertable_sql(expression)
|
||||
return super().alter_sql(expression)
|
||||
|
||||
def drop_sql(self, expression: exp.Drop) -> str:
|
||||
if expression.args["kind"] == "VIEW":
|
||||
|
|
|
@ -74,7 +74,9 @@ def execute(
|
|||
raise ExecuteError("Tables must support the same table args as schema")
|
||||
|
||||
now = time.time()
|
||||
expression = optimize(sql, schema, leave_tables_isolated=True, dialect=read)
|
||||
expression = optimize(
|
||||
sql, schema, leave_tables_isolated=True, infer_csv_schemas=True, dialect=read
|
||||
)
|
||||
|
||||
logger.debug("Optimization finished: %f", time.time() - now)
|
||||
logger.debug("Optimized SQL: %s", expression.sql(pretty=True))
|
||||
|
|
|
@ -1387,6 +1387,7 @@ class Create(DDL):
|
|||
"exists": False,
|
||||
"properties": False,
|
||||
"replace": False,
|
||||
"refresh": False,
|
||||
"unique": False,
|
||||
"indexes": False,
|
||||
"no_schema_binding": False,
|
||||
|
@ -1436,7 +1437,13 @@ class Clone(Expression):
|
|||
|
||||
|
||||
class Describe(Expression):
|
||||
arg_types = {"this": True, "style": False, "kind": False, "expressions": False}
|
||||
arg_types = {
|
||||
"this": True,
|
||||
"style": False,
|
||||
"kind": False,
|
||||
"expressions": False,
|
||||
"partition": False,
|
||||
}
|
||||
|
||||
|
||||
# https://duckdb.org/docs/guides/meta/summarize.html
|
||||
|
@ -2000,6 +2007,11 @@ class Drop(Expression):
|
|||
"cluster": False,
|
||||
}
|
||||
|
||||
@property
|
||||
def kind(self) -> t.Optional[str]:
|
||||
kind = self.args.get("kind")
|
||||
return kind and kind.upper()
|
||||
|
||||
|
||||
class Filter(Expression):
|
||||
arg_types = {"this": True, "expression": True}
|
||||
|
@ -2158,6 +2170,8 @@ class Insert(DDL, DML):
|
|||
"ignore": False,
|
||||
"by_name": False,
|
||||
"stored": False,
|
||||
"partition": False,
|
||||
"settings": False,
|
||||
}
|
||||
|
||||
def with_(
|
||||
|
@ -2464,17 +2478,17 @@ class Offset(Expression):
|
|||
|
||||
|
||||
class Order(Expression):
|
||||
arg_types = {
|
||||
"this": False,
|
||||
"expressions": True,
|
||||
"interpolate": False,
|
||||
"siblings": False,
|
||||
}
|
||||
arg_types = {"this": False, "expressions": True, "siblings": False}
|
||||
|
||||
|
||||
# https://clickhouse.com/docs/en/sql-reference/statements/select/order-by#order-by-expr-with-fill-modifier
|
||||
class WithFill(Expression):
|
||||
arg_types = {"from": False, "to": False, "step": False}
|
||||
arg_types = {
|
||||
"from": False,
|
||||
"to": False,
|
||||
"step": False,
|
||||
"interpolate": False,
|
||||
}
|
||||
|
||||
|
||||
# hive specific sorts
|
||||
|
@ -2669,6 +2683,11 @@ class OnCluster(Property):
|
|||
arg_types = {"this": True}
|
||||
|
||||
|
||||
# Clickhouse EMPTY table "property"
|
||||
class EmptyProperty(Property):
|
||||
arg_types = {}
|
||||
|
||||
|
||||
class LikeProperty(Property):
|
||||
arg_types = {"this": True, "expressions": False}
|
||||
|
||||
|
@ -2735,6 +2754,10 @@ class PartitionedOfProperty(Property):
|
|||
arg_types = {"this": True, "expression": True}
|
||||
|
||||
|
||||
class StreamingTableProperty(Property):
|
||||
arg_types = {}
|
||||
|
||||
|
||||
class RemoteWithConnectionModelProperty(Property):
|
||||
arg_types = {"this": True}
|
||||
|
||||
|
@ -3137,11 +3160,11 @@ class SetOperation(Query):
|
|||
return self.this.unnest().selects
|
||||
|
||||
@property
|
||||
def left(self) -> Expression:
|
||||
def left(self) -> Query:
|
||||
return self.this
|
||||
|
||||
@property
|
||||
def right(self) -> Expression:
|
||||
def right(self) -> Query:
|
||||
return self.expression
|
||||
|
||||
|
||||
|
@ -3859,6 +3882,7 @@ class Pivot(Expression):
|
|||
"group": False,
|
||||
"columns": False,
|
||||
"include_nulls": False,
|
||||
"default_on_null": False,
|
||||
}
|
||||
|
||||
@property
|
||||
|
@ -3948,6 +3972,8 @@ class DataTypeParam(Expression):
|
|||
return self.this.name
|
||||
|
||||
|
||||
# The `nullable` arg is helpful when transpiling types from other dialects to ClickHouse, which
|
||||
# assumes non-nullable types by default. Values `None` and `True` mean the type is nullable.
|
||||
class DataType(Expression):
|
||||
arg_types = {
|
||||
"this": True,
|
||||
|
@ -3956,6 +3982,7 @@ class DataType(Expression):
|
|||
"values": False,
|
||||
"prefix": False,
|
||||
"kind": False,
|
||||
"nullable": False,
|
||||
}
|
||||
|
||||
class Type(AutoName):
|
||||
|
@ -4194,28 +4221,45 @@ class DataType(Expression):
|
|||
|
||||
return DataType(**{**data_type_exp.args, **kwargs})
|
||||
|
||||
def is_type(self, *dtypes: DATA_TYPE) -> bool:
|
||||
def is_type(self, *dtypes: DATA_TYPE, check_nullable: bool = False) -> bool:
|
||||
"""
|
||||
Checks whether this DataType matches one of the provided data types. Nested types or precision
|
||||
will be compared using "structural equivalence" semantics, so e.g. array<int> != array<float>.
|
||||
|
||||
Args:
|
||||
dtypes: the data types to compare this DataType to.
|
||||
check_nullable: whether to take the NULLABLE type constructor into account for the comparison.
|
||||
If false, it means that NULLABLE<INT> is equivalent to INT.
|
||||
|
||||
Returns:
|
||||
True, if and only if there is a type in `dtypes` which is equal to this DataType.
|
||||
"""
|
||||
if (
|
||||
not check_nullable
|
||||
and self.this == DataType.Type.NULLABLE
|
||||
and len(self.expressions) == 1
|
||||
):
|
||||
this_type = self.expressions[0]
|
||||
else:
|
||||
this_type = self
|
||||
|
||||
for dtype in dtypes:
|
||||
other = DataType.build(dtype, copy=False, udt=True)
|
||||
other_type = DataType.build(dtype, copy=False, udt=True)
|
||||
if (
|
||||
not check_nullable
|
||||
and other_type.this == DataType.Type.NULLABLE
|
||||
and len(other_type.expressions) == 1
|
||||
):
|
||||
other_type = other_type.expressions[0]
|
||||
|
||||
if (
|
||||
other.expressions
|
||||
or self.this == DataType.Type.USERDEFINED
|
||||
or other.this == DataType.Type.USERDEFINED
|
||||
other_type.expressions
|
||||
or this_type.this == DataType.Type.USERDEFINED
|
||||
or other_type.this == DataType.Type.USERDEFINED
|
||||
):
|
||||
matches = self == other
|
||||
matches = this_type == other_type
|
||||
else:
|
||||
matches = self.this == other.this
|
||||
matches = this_type.this == other_type.this
|
||||
|
||||
if matches:
|
||||
return True
|
||||
|
@ -4270,9 +4314,10 @@ class Rollback(Expression):
|
|||
arg_types = {"savepoint": False, "this": False}
|
||||
|
||||
|
||||
class AlterTable(Expression):
|
||||
class Alter(Expression):
|
||||
arg_types = {
|
||||
"this": True,
|
||||
"kind": True,
|
||||
"actions": True,
|
||||
"exists": False,
|
||||
"only": False,
|
||||
|
@ -4536,6 +4581,12 @@ class PivotAlias(Alias):
|
|||
pass
|
||||
|
||||
|
||||
# Represents Snowflake's ANY [ ORDER BY ... ] syntax
|
||||
# https://docs.snowflake.com/en/sql-reference/constructs/pivot
|
||||
class PivotAny(Expression):
|
||||
arg_types = {"this": False}
|
||||
|
||||
|
||||
class Aliases(Expression):
|
||||
arg_types = {"this": True, "expressions": True}
|
||||
|
||||
|
@ -4790,7 +4841,7 @@ class ApproxDistinct(AggFunc):
|
|||
|
||||
|
||||
class Array(Func):
|
||||
arg_types = {"expressions": False}
|
||||
arg_types = {"expressions": False, "bracket_notation": False}
|
||||
is_var_len_args = True
|
||||
|
||||
|
||||
|
@ -4833,10 +4884,21 @@ class Convert(Func):
|
|||
arg_types = {"this": True, "expression": True, "style": False}
|
||||
|
||||
|
||||
class ConvertTimezone(Func):
|
||||
arg_types = {"source_tz": False, "target_tz": True, "timestamp": True}
|
||||
|
||||
|
||||
class GenerateSeries(Func):
|
||||
arg_types = {"start": True, "end": True, "step": False, "is_end_exclusive": False}
|
||||
|
||||
|
||||
# Postgres' GENERATE_SERIES function returns a row set, i.e. it implicitly explodes when it's
|
||||
# used in a projection, so this expression is a helper that facilitates transpilation to other
|
||||
# dialects. For example, we'd generate UNNEST(GENERATE_SERIES(...)) in DuckDB
|
||||
class ExplodingGenerateSeries(GenerateSeries):
|
||||
pass
|
||||
|
||||
|
||||
class ArrayAgg(AggFunc):
|
||||
pass
|
||||
|
||||
|
@ -5025,7 +5087,7 @@ class Ceil(Func):
|
|||
|
||||
|
||||
class Coalesce(Func):
|
||||
arg_types = {"this": True, "expressions": False}
|
||||
arg_types = {"this": True, "expressions": False, "is_nvl": False}
|
||||
is_var_len_args = True
|
||||
_sql_names = ["COALESCE", "IFNULL", "NVL"]
|
||||
|
||||
|
@ -5077,7 +5139,7 @@ class CurrentTime(Func):
|
|||
|
||||
|
||||
class CurrentTimestamp(Func):
|
||||
arg_types = {"this": False, "transaction": False}
|
||||
arg_types = {"this": False, "sysdate": False}
|
||||
|
||||
|
||||
class CurrentUser(Func):
|
||||
|
@ -5286,6 +5348,7 @@ class Unnest(Func, UDTF):
|
|||
"expressions": True,
|
||||
"alias": False,
|
||||
"offset": False,
|
||||
"explode_array": False,
|
||||
}
|
||||
|
||||
@property
|
||||
|
@ -5309,6 +5372,11 @@ class ToBase64(Func):
|
|||
pass
|
||||
|
||||
|
||||
# https://trino.io/docs/current/functions/datetime.html#from_iso8601_timestamp
|
||||
class FromISO8601Timestamp(Func):
|
||||
_sql_names = ["FROM_ISO8601_TIMESTAMP"]
|
||||
|
||||
|
||||
class GapFill(Func):
|
||||
arg_types = {
|
||||
"this": True,
|
||||
|
@ -5321,8 +5389,14 @@ class GapFill(Func):
|
|||
}
|
||||
|
||||
|
||||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/array_functions#generate_date_array
|
||||
class GenerateDateArray(Func):
|
||||
arg_types = {"start": True, "end": True, "interval": False}
|
||||
arg_types = {"start": True, "end": True, "step": False}
|
||||
|
||||
|
||||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/array_functions#generate_timestamp_array
|
||||
class GenerateTimestampArray(Func):
|
||||
arg_types = {"start": True, "end": True, "step": True}
|
||||
|
||||
|
||||
class Greatest(Func):
|
||||
|
@ -5639,6 +5713,10 @@ class ScopeResolution(Expression):
|
|||
arg_types = {"this": False, "expression": True}
|
||||
|
||||
|
||||
class Stream(Expression):
|
||||
pass
|
||||
|
||||
|
||||
class StarMap(Func):
|
||||
pass
|
||||
|
||||
|
@ -5920,7 +5998,7 @@ class Time(Func):
|
|||
|
||||
|
||||
class TimeToStr(Func):
|
||||
arg_types = {"this": True, "format": True, "culture": False, "timezone": False}
|
||||
arg_types = {"this": True, "format": True, "culture": False, "zone": False}
|
||||
|
||||
|
||||
class TimeToTimeStr(Func):
|
||||
|
@ -5936,7 +6014,7 @@ class TimeStrToDate(Func):
|
|||
|
||||
|
||||
class TimeStrToTime(Func):
|
||||
pass
|
||||
arg_types = {"this": True, "zone": False}
|
||||
|
||||
|
||||
class TimeStrToUnix(Func):
|
||||
|
@ -7144,7 +7222,9 @@ def column(
|
|||
return this
|
||||
|
||||
|
||||
def cast(expression: ExpOrStr, to: DATA_TYPE, copy: bool = True, **opts) -> Cast:
|
||||
def cast(
|
||||
expression: ExpOrStr, to: DATA_TYPE, copy: bool = True, dialect: DialectType = None, **opts
|
||||
) -> Cast:
|
||||
"""Cast an expression to a data type.
|
||||
|
||||
Example:
|
||||
|
@ -7155,14 +7235,36 @@ def cast(expression: ExpOrStr, to: DATA_TYPE, copy: bool = True, **opts) -> Cast
|
|||
expression: The expression to cast.
|
||||
to: The datatype to cast to.
|
||||
copy: Whether to copy the supplied expressions.
|
||||
dialect: The target dialect. This is used to prevent a re-cast in the following scenario:
|
||||
- The expression to be cast is already a exp.Cast expression
|
||||
- The existing cast is to a type that is logically equivalent to new type
|
||||
|
||||
For example, if :expression='CAST(x as DATETIME)' and :to=Type.TIMESTAMP,
|
||||
but in the target dialect DATETIME is mapped to TIMESTAMP, then we will NOT return `CAST(x (as DATETIME) as TIMESTAMP)`
|
||||
and instead just return the original expression `CAST(x as DATETIME)`.
|
||||
|
||||
This is to prevent it being output as a double cast `CAST(x (as TIMESTAMP) as TIMESTAMP)` once the DATETIME -> TIMESTAMP
|
||||
mapping is applied in the target dialect generator.
|
||||
|
||||
Returns:
|
||||
The new Cast instance.
|
||||
"""
|
||||
expr = maybe_parse(expression, copy=copy, **opts)
|
||||
data_type = DataType.build(to, copy=copy, **opts)
|
||||
expr = maybe_parse(expression, copy=copy, dialect=dialect, **opts)
|
||||
data_type = DataType.build(to, copy=copy, dialect=dialect, **opts)
|
||||
|
||||
if expr.is_type(data_type):
|
||||
# dont re-cast if the expression is already a cast to the correct type
|
||||
if isinstance(expr, Cast):
|
||||
from sqlglot.dialects.dialect import Dialect
|
||||
|
||||
target_dialect = Dialect.get_or_raise(dialect)
|
||||
type_mapping = target_dialect.generator_class.TYPE_MAPPING
|
||||
|
||||
existing_cast_type: DataType.Type = expr.to.this
|
||||
new_cast_type: DataType.Type = data_type.this
|
||||
types_are_equivalent = type_mapping.get(
|
||||
existing_cast_type, existing_cast_type
|
||||
) == type_mapping.get(new_cast_type, new_cast_type)
|
||||
if expr.is_type(data_type) or types_are_equivalent:
|
||||
return expr
|
||||
|
||||
expr = Cast(this=expr, to=data_type)
|
||||
|
@ -7259,7 +7361,7 @@ def rename_table(
|
|||
old_name: str | Table,
|
||||
new_name: str | Table,
|
||||
dialect: DialectType = None,
|
||||
) -> AlterTable:
|
||||
) -> Alter:
|
||||
"""Build ALTER TABLE... RENAME... expression
|
||||
|
||||
Args:
|
||||
|
@ -7272,8 +7374,9 @@ def rename_table(
|
|||
"""
|
||||
old_table = to_table(old_name, dialect=dialect)
|
||||
new_table = to_table(new_name, dialect=dialect)
|
||||
return AlterTable(
|
||||
return Alter(
|
||||
this=old_table,
|
||||
kind="TABLE",
|
||||
actions=[
|
||||
RenameTable(this=new_table),
|
||||
],
|
||||
|
@ -7286,7 +7389,7 @@ def rename_column(
|
|||
new_column_name: str | Column,
|
||||
exists: t.Optional[bool] = None,
|
||||
dialect: DialectType = None,
|
||||
) -> AlterTable:
|
||||
) -> Alter:
|
||||
"""Build ALTER TABLE... RENAME COLUMN... expression
|
||||
|
||||
Args:
|
||||
|
@ -7302,8 +7405,9 @@ def rename_column(
|
|||
table = to_table(table_name, dialect=dialect)
|
||||
old_column = to_column(old_column_name, dialect=dialect)
|
||||
new_column = to_column(new_column_name, dialect=dialect)
|
||||
return AlterTable(
|
||||
return Alter(
|
||||
this=table,
|
||||
kind="TABLE",
|
||||
actions=[
|
||||
RenameColumn(this=old_column, to=new_column, exists=exists),
|
||||
],
|
||||
|
@ -7335,12 +7439,15 @@ def convert(value: t.Any, copy: bool = False) -> Expression:
|
|||
if isinstance(value, bytes):
|
||||
return HexString(this=value.hex())
|
||||
if isinstance(value, datetime.datetime):
|
||||
datetime_literal = Literal.string(
|
||||
(value if value.tzinfo else value.replace(tzinfo=datetime.timezone.utc)).isoformat(
|
||||
sep=" "
|
||||
)
|
||||
)
|
||||
return TimeStrToTime(this=datetime_literal)
|
||||
datetime_literal = Literal.string(value.isoformat(sep=" "))
|
||||
|
||||
tz = None
|
||||
if value.tzinfo:
|
||||
# this works for zoneinfo.ZoneInfo, pytz.timezone and datetime.datetime.utc to return IANA timezone names like "America/Los_Angeles"
|
||||
# instead of abbreviations like "PDT". This is for consistency with other timezone handling functions in SQLGlot
|
||||
tz = Literal.string(str(value.tzinfo))
|
||||
|
||||
return TimeStrToTime(this=datetime_literal, zone=tz)
|
||||
if isinstance(value, datetime.date):
|
||||
date_literal = Literal.string(value.strftime("%Y-%m-%d"))
|
||||
return DateStrToDate(this=date_literal)
|
||||
|
|
|
@ -92,6 +92,7 @@ class Generator(metaclass=_Generator):
|
|||
exp.DateFormatColumnConstraint: lambda self, e: f"FORMAT {self.sql(e, 'this')}",
|
||||
exp.DefaultColumnConstraint: lambda self, e: f"DEFAULT {self.sql(e, 'this')}",
|
||||
exp.DynamicProperty: lambda *_: "DYNAMIC",
|
||||
exp.EmptyProperty: lambda *_: "EMPTY",
|
||||
exp.EncodeColumnConstraint: lambda self, e: f"ENCODE {self.sql(e, 'this')}",
|
||||
exp.EphemeralColumnConstraint: lambda self,
|
||||
e: f"EPHEMERAL{(' ' + self.sql(e, 'this')) if e.this else ''}",
|
||||
|
@ -117,8 +118,10 @@ class Generator(metaclass=_Generator):
|
|||
e: f"ON COMMIT {'DELETE' if e.args.get('delete') else 'PRESERVE'} ROWS",
|
||||
exp.OnProperty: lambda self, e: f"ON {self.sql(e, 'this')}",
|
||||
exp.OnUpdateColumnConstraint: lambda self, e: f"ON UPDATE {self.sql(e, 'this')}",
|
||||
exp.Operator: lambda self, e: self.binary(e, ""), # The operator is produced in `binary`
|
||||
exp.OutputModelProperty: lambda self, e: f"OUTPUT{self.sql(e, 'this')}",
|
||||
exp.PathColumnConstraint: lambda self, e: f"PATH {self.sql(e, 'this')}",
|
||||
exp.PivotAny: lambda self, e: f"ANY{self.sql(e, 'this')}",
|
||||
exp.ProjectionPolicyColumnConstraint: lambda self,
|
||||
e: f"PROJECTION POLICY {self.sql(e, 'this')}",
|
||||
exp.RemoteWithConnectionModelProperty: lambda self,
|
||||
|
@ -136,6 +139,8 @@ class Generator(metaclass=_Generator):
|
|||
exp.SqlSecurityProperty: lambda _,
|
||||
e: f"SQL SECURITY {'DEFINER' if e.args.get('definer') else 'INVOKER'}",
|
||||
exp.StabilityProperty: lambda _, e: e.name,
|
||||
exp.Stream: lambda self, e: f"STREAM {self.sql(e, 'this')}",
|
||||
exp.StreamingTableProperty: lambda *_: "STREAMING",
|
||||
exp.StrictProperty: lambda *_: "STRICT",
|
||||
exp.TemporaryProperty: lambda *_: "TEMPORARY",
|
||||
exp.TagColumnConstraint: lambda self, e: f"TAG ({self.expressions(e, flat=True)})",
|
||||
|
@ -371,6 +376,18 @@ class Generator(metaclass=_Generator):
|
|||
# Whether the text pattern/fill (3rd) parameter of RPAD()/LPAD() is optional (defaults to space)
|
||||
PAD_FILL_PATTERN_IS_REQUIRED = False
|
||||
|
||||
# Whether a projection can explode into multiple rows, e.g. by unnesting an array.
|
||||
SUPPORTS_EXPLODING_PROJECTIONS = True
|
||||
|
||||
# Whether ARRAY_CONCAT can be generated with varlen args or if it should be reduced to 2-arg version
|
||||
ARRAY_CONCAT_IS_VAR_LEN = True
|
||||
|
||||
# Whether CONVERT_TIMEZONE() is supported; if not, it will be generated as exp.AtTimeZone
|
||||
SUPPORTS_CONVERT_TIMEZONE = False
|
||||
|
||||
# Whether nullable types can be constructed, e.g. `Nullable(Int64)`
|
||||
SUPPORTS_NULLABLE_TYPES = True
|
||||
|
||||
# The name to generate for the JSONPath expression. If `None`, only `this` will be generated
|
||||
PARSE_JSON_NAME: t.Optional[str] = "PARSE_JSON"
|
||||
|
||||
|
@ -439,6 +456,7 @@ class Generator(metaclass=_Generator):
|
|||
exp.DynamicProperty: exp.Properties.Location.POST_CREATE,
|
||||
exp.DistKeyProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.DistStyleProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.EmptyProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.EngineProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.ExecuteAsProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.ExternalProperty: exp.Properties.Location.POST_CREATE,
|
||||
|
@ -488,6 +506,7 @@ class Generator(metaclass=_Generator):
|
|||
exp.SqlReadWriteProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.SqlSecurityProperty: exp.Properties.Location.POST_CREATE,
|
||||
exp.StabilityProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.StreamingTableProperty: exp.Properties.Location.POST_CREATE,
|
||||
exp.StrictProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.TemporaryProperty: exp.Properties.Location.POST_CREATE,
|
||||
exp.ToTableProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
|
@ -962,6 +981,7 @@ class Generator(metaclass=_Generator):
|
|||
|
||||
def create_sql(self, expression: exp.Create) -> str:
|
||||
kind = self.sql(expression, "kind")
|
||||
kind = self.dialect.INVERSE_CREATABLE_KIND_MAPPING.get(kind) or kind
|
||||
properties = expression.args.get("properties")
|
||||
properties_locs = self.locate_properties(properties) if properties else defaultdict()
|
||||
|
||||
|
@ -1018,6 +1038,7 @@ class Generator(metaclass=_Generator):
|
|||
index_sql = indexes + postindex_props_sql
|
||||
|
||||
replace = " OR REPLACE" if expression.args.get("replace") else ""
|
||||
refresh = " OR REFRESH" if expression.args.get("refresh") else ""
|
||||
unique = " UNIQUE" if expression.args.get("unique") else ""
|
||||
|
||||
clustered = expression.args.get("clustered")
|
||||
|
@ -1037,7 +1058,7 @@ class Generator(metaclass=_Generator):
|
|||
wrapped=False,
|
||||
)
|
||||
|
||||
modifiers = "".join((clustered_sql, replace, unique, postcreate_props_sql))
|
||||
modifiers = "".join((clustered_sql, replace, refresh, unique, postcreate_props_sql))
|
||||
|
||||
postexpression_props_sql = ""
|
||||
if properties_locs.get(exp.Properties.Location.POST_EXPRESSION):
|
||||
|
@ -1096,7 +1117,9 @@ class Generator(metaclass=_Generator):
|
|||
def describe_sql(self, expression: exp.Describe) -> str:
|
||||
style = expression.args.get("style")
|
||||
style = f" {style}" if style else ""
|
||||
return f"DESCRIBE{style} {self.sql(expression, 'this')}"
|
||||
partition = self.sql(expression, "partition")
|
||||
partition = f" {partition}" if partition else ""
|
||||
return f"DESCRIBE{style} {self.sql(expression, 'this')}{partition}"
|
||||
|
||||
def heredoc_sql(self, expression: exp.Heredoc) -> str:
|
||||
tag = self.sql(expression, "tag")
|
||||
|
@ -1195,20 +1218,21 @@ class Generator(metaclass=_Generator):
|
|||
return f"{this}{specifier}"
|
||||
|
||||
def datatype_sql(self, expression: exp.DataType) -> str:
|
||||
type_value = expression.this
|
||||
nested = ""
|
||||
values = ""
|
||||
interior = self.expressions(expression, flat=True)
|
||||
|
||||
type_value = expression.this
|
||||
if type_value == exp.DataType.Type.USERDEFINED and expression.args.get("kind"):
|
||||
type_sql = self.sql(expression, "kind")
|
||||
else:
|
||||
elif type_value != exp.DataType.Type.NULLABLE or self.SUPPORTS_NULLABLE_TYPES:
|
||||
type_sql = (
|
||||
self.TYPE_MAPPING.get(type_value, type_value.value)
|
||||
if isinstance(type_value, exp.DataType.Type)
|
||||
else type_value
|
||||
)
|
||||
|
||||
nested = ""
|
||||
interior = self.expressions(expression, flat=True)
|
||||
values = ""
|
||||
else:
|
||||
return interior
|
||||
|
||||
if interior:
|
||||
if expression.args.get("nested"):
|
||||
|
@ -1258,6 +1282,7 @@ class Generator(metaclass=_Generator):
|
|||
expressions = self.expressions(expression, flat=True)
|
||||
expressions = f" ({expressions})" if expressions else ""
|
||||
kind = expression.args["kind"]
|
||||
kind = self.dialect.INVERSE_CREATABLE_KIND_MAPPING.get(kind) or kind
|
||||
exists_sql = " IF EXISTS " if expression.args.get("exists") else " "
|
||||
on_cluster = self.sql(expression, "cluster")
|
||||
on_cluster = f" {on_cluster}" if on_cluster else ""
|
||||
|
@ -1277,7 +1302,7 @@ class Generator(metaclass=_Generator):
|
|||
def fetch_sql(self, expression: exp.Fetch) -> str:
|
||||
direction = expression.args.get("direction")
|
||||
direction = f" {direction}" if direction else ""
|
||||
count = expression.args.get("count")
|
||||
count = self.sql(expression, "count")
|
||||
count = f" {count}" if count else ""
|
||||
if expression.args.get("percent"):
|
||||
count = f"{count} PERCENT"
|
||||
|
@ -1639,7 +1664,12 @@ class Generator(metaclass=_Generator):
|
|||
else:
|
||||
expression_sql = f"{returning}{expression_sql}{on_conflict}"
|
||||
|
||||
sql = f"INSERT{hint}{alternative}{ignore}{this}{stored}{by_name}{exists}{where}{expression_sql}"
|
||||
partition_by = self.sql(expression, "partition")
|
||||
partition_by = f" {partition_by}" if partition_by else ""
|
||||
settings = self.sql(expression, "settings")
|
||||
settings = f" {settings}" if settings else ""
|
||||
|
||||
sql = f"INSERT{hint}{alternative}{ignore}{this}{stored}{by_name}{exists}{partition_by}{settings}{where}{expression_sql}"
|
||||
return self.prepend_ctes(expression, sql)
|
||||
|
||||
def intersect_sql(self, expression: exp.Intersect) -> str:
|
||||
|
@ -1824,13 +1854,20 @@ class Generator(metaclass=_Generator):
|
|||
alias = self.sql(expression, "alias")
|
||||
alias = f" AS {alias}" if alias else ""
|
||||
direction = self.seg("UNPIVOT" if expression.unpivot else "PIVOT")
|
||||
|
||||
field = self.sql(expression, "field")
|
||||
if field and isinstance(expression.args.get("field"), exp.PivotAny):
|
||||
field = f"IN ({field})"
|
||||
|
||||
include_nulls = expression.args.get("include_nulls")
|
||||
if include_nulls is not None:
|
||||
nulls = " INCLUDE NULLS " if include_nulls else " EXCLUDE NULLS "
|
||||
else:
|
||||
nulls = ""
|
||||
return f"{direction}{nulls}({expressions} FOR {field}){alias}"
|
||||
|
||||
default_on_null = self.sql(expression, "default_on_null")
|
||||
default_on_null = f" DEFAULT ON NULL ({default_on_null})" if default_on_null else ""
|
||||
return f"{direction}{nulls}({expressions} FOR {field}{default_on_null}){alias}"
|
||||
|
||||
def version_sql(self, expression: exp.Version) -> str:
|
||||
this = f"FOR {expression.name}"
|
||||
|
@ -2148,15 +2185,7 @@ class Generator(metaclass=_Generator):
|
|||
this = self.sql(expression, "this")
|
||||
this = f"{this} " if this else this
|
||||
siblings = "SIBLINGS " if expression.args.get("siblings") else ""
|
||||
order = self.op_expressions(f"{this}ORDER {siblings}BY", expression, flat=this or flat) # type: ignore
|
||||
interpolated_values = [
|
||||
f"{self.sql(named_expression, 'alias')} AS {self.sql(named_expression, 'this')}"
|
||||
for named_expression in expression.args.get("interpolate") or []
|
||||
]
|
||||
interpolate = (
|
||||
f" INTERPOLATE ({', '.join(interpolated_values)})" if interpolated_values else ""
|
||||
)
|
||||
return f"{order}{interpolate}"
|
||||
return self.op_expressions(f"{this}ORDER {siblings}BY", expression, flat=this or flat) # type: ignore
|
||||
|
||||
def withfill_sql(self, expression: exp.WithFill) -> str:
|
||||
from_sql = self.sql(expression, "from")
|
||||
|
@ -2165,7 +2194,14 @@ class Generator(metaclass=_Generator):
|
|||
to_sql = f" TO {to_sql}" if to_sql else ""
|
||||
step_sql = self.sql(expression, "step")
|
||||
step_sql = f" STEP {step_sql}" if step_sql else ""
|
||||
return f"WITH FILL{from_sql}{to_sql}{step_sql}"
|
||||
interpolated_values = [
|
||||
f"{self.sql(named_expression, 'alias')} AS {self.sql(named_expression, 'this')}"
|
||||
for named_expression in expression.args.get("interpolate") or []
|
||||
]
|
||||
interpolate = (
|
||||
f" INTERPOLATE ({', '.join(interpolated_values)})" if interpolated_values else ""
|
||||
)
|
||||
return f"WITH FILL{from_sql}{to_sql}{step_sql}{interpolate}"
|
||||
|
||||
def cluster_sql(self, expression: exp.Cluster) -> str:
|
||||
return self.op_expressions("CLUSTER BY", expression)
|
||||
|
@ -2875,11 +2911,13 @@ class Generator(metaclass=_Generator):
|
|||
|
||||
def pivotalias_sql(self, expression: exp.PivotAlias) -> str:
|
||||
alias = expression.args["alias"]
|
||||
|
||||
identifier_alias = isinstance(alias, exp.Identifier)
|
||||
literal_alias = isinstance(alias, exp.Literal)
|
||||
|
||||
if identifier_alias and not self.UNPIVOT_ALIASES_ARE_IDENTIFIERS:
|
||||
alias.replace(exp.Literal.string(alias.output_name))
|
||||
elif not identifier_alias and self.UNPIVOT_ALIASES_ARE_IDENTIFIERS:
|
||||
elif not identifier_alias and literal_alias and self.UNPIVOT_ALIASES_ARE_IDENTIFIERS:
|
||||
alias.replace(exp.to_identifier(alias.output_name))
|
||||
|
||||
return self.alias_sql(expression)
|
||||
|
@ -3103,7 +3141,7 @@ class Generator(metaclass=_Generator):
|
|||
exprs = self.expressions(expression, flat=True)
|
||||
return f"SET {exprs}"
|
||||
|
||||
def altertable_sql(self, expression: exp.AlterTable) -> str:
|
||||
def alter_sql(self, expression: exp.Alter) -> str:
|
||||
actions = expression.args["actions"]
|
||||
|
||||
if isinstance(actions[0], exp.ColumnDef):
|
||||
|
@ -3112,6 +3150,8 @@ class Generator(metaclass=_Generator):
|
|||
actions = self.expressions(expression, key="actions", prefix="ADD COLUMNS ")
|
||||
elif isinstance(actions[0], exp.Delete):
|
||||
actions = self.expressions(expression, key="actions", flat=True)
|
||||
elif isinstance(actions[0], exp.Query):
|
||||
actions = "AS " + self.expressions(expression, key="actions")
|
||||
else:
|
||||
actions = self.expressions(expression, key="actions", flat=True)
|
||||
|
||||
|
@ -3121,9 +3161,10 @@ class Generator(metaclass=_Generator):
|
|||
only = " ONLY" if expression.args.get("only") else ""
|
||||
options = self.expressions(expression, key="options")
|
||||
options = f", {options}" if options else ""
|
||||
return f"ALTER TABLE{exists}{only} {self.sql(expression, 'this')}{on_cluster} {actions}{options}"
|
||||
kind = self.sql(expression, "kind")
|
||||
return f"ALTER {kind}{exists}{only} {self.sql(expression, 'this')}{on_cluster} {actions}{options}"
|
||||
|
||||
def add_column_sql(self, expression: exp.AlterTable) -> str:
|
||||
def add_column_sql(self, expression: exp.Alter) -> str:
|
||||
if self.ALTER_TABLE_INCLUDE_COLUMN_KEYWORD:
|
||||
return self.expressions(
|
||||
expression,
|
||||
|
@ -3312,8 +3353,25 @@ class Generator(metaclass=_Generator):
|
|||
return f"USE{kind}{this}"
|
||||
|
||||
def binary(self, expression: exp.Binary, op: str) -> str:
|
||||
op = self.maybe_comment(op, comments=expression.comments)
|
||||
return f"{self.sql(expression, 'this')} {op} {self.sql(expression, 'expression')}"
|
||||
sqls: t.List[str] = []
|
||||
stack: t.List[t.Union[str, exp.Expression]] = [expression]
|
||||
binary_type = type(expression)
|
||||
|
||||
while stack:
|
||||
node = stack.pop()
|
||||
|
||||
if type(node) is binary_type:
|
||||
op_func = node.args.get("operator")
|
||||
if op_func:
|
||||
op = f"OPERATOR({self.sql(op_func)})"
|
||||
|
||||
stack.append(node.right)
|
||||
stack.append(f" {self.maybe_comment(op, comments=node.comments)} ")
|
||||
stack.append(node.left)
|
||||
else:
|
||||
sqls.append(self.sql(node))
|
||||
|
||||
return "".join(sqls)
|
||||
|
||||
def function_fallback_sql(self, expression: exp.Func) -> str:
|
||||
args = []
|
||||
|
@ -3660,9 +3718,6 @@ class Generator(metaclass=_Generator):
|
|||
table = "" if isinstance(expression.this, exp.Literal) else "TABLE "
|
||||
return f"REFRESH {table}{this}"
|
||||
|
||||
def operator_sql(self, expression: exp.Operator) -> str:
|
||||
return self.binary(expression, f"OPERATOR({self.sql(expression, 'operator')})")
|
||||
|
||||
def toarray_sql(self, expression: exp.ToArray) -> str:
|
||||
arg = expression.this
|
||||
if not arg.type:
|
||||
|
@ -4041,3 +4096,44 @@ class Generator(metaclass=_Generator):
|
|||
def summarize_sql(self, expression: exp.Summarize) -> str:
|
||||
table = " TABLE" if expression.args.get("table") else ""
|
||||
return f"SUMMARIZE{table} {self.sql(expression.this)}"
|
||||
|
||||
def explodinggenerateseries_sql(self, expression: exp.ExplodingGenerateSeries) -> str:
|
||||
generate_series = exp.GenerateSeries(**expression.args)
|
||||
|
||||
parent = expression.parent
|
||||
if isinstance(parent, (exp.Alias, exp.TableAlias)):
|
||||
parent = parent.parent
|
||||
|
||||
if self.SUPPORTS_EXPLODING_PROJECTIONS and not isinstance(parent, (exp.Table, exp.Unnest)):
|
||||
return self.sql(exp.Unnest(expressions=[generate_series]))
|
||||
|
||||
if isinstance(parent, exp.Select):
|
||||
self.unsupported("GenerateSeries projection unnesting is not supported.")
|
||||
|
||||
return self.sql(generate_series)
|
||||
|
||||
def arrayconcat_sql(self, expression: exp.ArrayConcat, name: str = "ARRAY_CONCAT") -> str:
|
||||
exprs = expression.expressions
|
||||
if not self.ARRAY_CONCAT_IS_VAR_LEN:
|
||||
rhs = reduce(lambda x, y: exp.ArrayConcat(this=x, expressions=[y]), exprs)
|
||||
else:
|
||||
rhs = self.expressions(expression)
|
||||
|
||||
return self.func(name, expression.this, rhs)
|
||||
|
||||
def converttimezone_sql(self, expression: exp.ConvertTimezone) -> str:
|
||||
if self.SUPPORTS_CONVERT_TIMEZONE:
|
||||
return self.function_fallback_sql(expression)
|
||||
|
||||
source_tz = expression.args.get("source_tz")
|
||||
target_tz = expression.args.get("target_tz")
|
||||
timestamp = expression.args.get("timestamp")
|
||||
|
||||
if source_tz and timestamp:
|
||||
timestamp = exp.AtTimeZone(
|
||||
this=exp.cast(timestamp, exp.DataType.Type.TIMESTAMPNTZ), zone=source_tz
|
||||
)
|
||||
|
||||
expr = exp.AtTimeZone(this=timestamp, zone=target_tz)
|
||||
|
||||
return self.sql(expr)
|
||||
|
|
|
@ -128,6 +128,13 @@ class _TypeAnnotator(type):
|
|||
klass.COERCES_TO[data_type] = coerces_to.copy()
|
||||
coerces_to |= {data_type}
|
||||
|
||||
# NULL can be coerced to any type, so e.g. NULL + 1 will have type INT
|
||||
klass.COERCES_TO[exp.DataType.Type.NULL] = {
|
||||
*text_precedence,
|
||||
*numeric_precedence,
|
||||
*timelike_precedence,
|
||||
}
|
||||
|
||||
return klass
|
||||
|
||||
|
||||
|
@ -201,31 +208,47 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
|||
for name, source in scope.sources.items():
|
||||
if not isinstance(source, Scope):
|
||||
continue
|
||||
if isinstance(source.expression, exp.UDTF):
|
||||
|
||||
expression = source.expression
|
||||
if isinstance(expression, exp.UDTF):
|
||||
values = []
|
||||
|
||||
if isinstance(source.expression, exp.Lateral):
|
||||
if isinstance(source.expression.this, exp.Explode):
|
||||
values = [source.expression.this.this]
|
||||
elif isinstance(source.expression, exp.Unnest):
|
||||
values = [source.expression]
|
||||
if isinstance(expression, exp.Lateral):
|
||||
if isinstance(expression.this, exp.Explode):
|
||||
values = [expression.this.this]
|
||||
elif isinstance(expression, exp.Unnest):
|
||||
values = [expression]
|
||||
else:
|
||||
values = source.expression.expressions[0].expressions
|
||||
values = expression.expressions[0].expressions
|
||||
|
||||
if not values:
|
||||
continue
|
||||
|
||||
selects[name] = {
|
||||
alias: column
|
||||
for alias, column in zip(
|
||||
source.expression.alias_column_names,
|
||||
values,
|
||||
alias: column.type
|
||||
for alias, column in zip(expression.alias_column_names, values)
|
||||
}
|
||||
elif isinstance(expression, exp.SetOperation) and len(expression.left.selects) == len(
|
||||
expression.right.selects
|
||||
):
|
||||
if expression.args.get("by_name"):
|
||||
r_type_by_select = {s.alias_or_name: s.type for s in expression.right.selects}
|
||||
selects[name] = {
|
||||
s.alias_or_name: self._maybe_coerce(
|
||||
t.cast(exp.DataType, s.type),
|
||||
r_type_by_select.get(s.alias_or_name) or exp.DataType.Type.UNKNOWN,
|
||||
)
|
||||
for s in expression.left.selects
|
||||
}
|
||||
else:
|
||||
selects[name] = {
|
||||
select.alias_or_name: select for select in source.expression.selects
|
||||
ls.alias_or_name: self._maybe_coerce(
|
||||
t.cast(exp.DataType, ls.type), t.cast(exp.DataType, rs.type)
|
||||
)
|
||||
for ls, rs in zip(expression.left.selects, expression.right.selects)
|
||||
}
|
||||
else:
|
||||
selects[name] = {s.alias_or_name: s.type for s in expression.selects}
|
||||
|
||||
# First annotate the current scope's column references
|
||||
for col in scope.columns:
|
||||
|
@ -237,7 +260,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
|||
self._set_type(col, self.schema.get_column_type(source, col))
|
||||
elif source:
|
||||
if col.table in selects and col.name in selects[col.table]:
|
||||
self._set_type(col, selects[col.table][col.name].type)
|
||||
self._set_type(col, selects[col.table][col.name])
|
||||
elif isinstance(source.expression, exp.Unnest):
|
||||
self._set_type(col, source.expression.type)
|
||||
|
||||
|
@ -264,15 +287,13 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
|||
|
||||
def _maybe_coerce(
|
||||
self, type1: exp.DataType | exp.DataType.Type, type2: exp.DataType | exp.DataType.Type
|
||||
) -> exp.DataType | exp.DataType.Type:
|
||||
) -> exp.DataType:
|
||||
type1_value = type1.this if isinstance(type1, exp.DataType) else type1
|
||||
type2_value = type2.this if isinstance(type2, exp.DataType) else type2
|
||||
|
||||
# We propagate the NULL / UNKNOWN types upwards if found
|
||||
if exp.DataType.Type.NULL in (type1_value, type2_value):
|
||||
return exp.DataType.Type.NULL
|
||||
# We propagate the UNKNOWN type upwards if found
|
||||
if exp.DataType.Type.UNKNOWN in (type1_value, type2_value):
|
||||
return exp.DataType.Type.UNKNOWN
|
||||
return exp.DataType.build("unknown")
|
||||
|
||||
return type2_value if type2_value in self.coerces_to.get(type1_value, {}) else type1_value
|
||||
|
||||
|
@ -282,17 +303,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
|||
left, right = expression.left, expression.right
|
||||
left_type, right_type = left.type.this, right.type.this # type: ignore
|
||||
|
||||
if isinstance(expression, exp.Connector):
|
||||
if left_type == exp.DataType.Type.NULL and right_type == exp.DataType.Type.NULL:
|
||||
self._set_type(expression, exp.DataType.Type.NULL)
|
||||
elif exp.DataType.Type.NULL in (left_type, right_type):
|
||||
self._set_type(
|
||||
expression,
|
||||
exp.DataType.build("NULLABLE", expressions=exp.DataType.build("BOOLEAN")),
|
||||
)
|
||||
else:
|
||||
self._set_type(expression, exp.DataType.Type.BOOLEAN)
|
||||
elif isinstance(expression, exp.Predicate):
|
||||
if isinstance(expression, (exp.Connector, exp.Predicate)):
|
||||
self._set_type(expression, exp.DataType.Type.BOOLEAN)
|
||||
elif (left_type, right_type) in self.binary_coercions:
|
||||
self._set_type(expression, self.binary_coercions[(left_type, right_type)](left, right))
|
||||
|
@ -351,7 +362,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
|||
last_datatype = expr_type
|
||||
break
|
||||
|
||||
if not expr_type.is_type(exp.DataType.Type.NULL, exp.DataType.Type.UNKNOWN):
|
||||
if not expr_type.is_type(exp.DataType.Type.UNKNOWN):
|
||||
last_datatype = self._maybe_coerce(last_datatype or expr_type, expr_type)
|
||||
|
||||
self._set_type(expression, last_datatype or exp.DataType.Type.UNKNOWN)
|
||||
|
|
|
@ -40,6 +40,8 @@ def replace_date_funcs(node: exp.Expression) -> exp.Expression:
|
|||
isinstance(node, (exp.Date, exp.TsOrDsToDate))
|
||||
and not node.expressions
|
||||
and not node.args.get("zone")
|
||||
and node.this.is_string
|
||||
and is_iso_date(node.this.name)
|
||||
):
|
||||
return exp.cast(node.this, to=exp.DataType.Type.DATE)
|
||||
if isinstance(node, exp.Timestamp) and not node.args.get("zone"):
|
||||
|
@ -90,6 +92,12 @@ def remove_redundant_casts(expression: exp.Expression) -> exp.Expression:
|
|||
and expression.to.this == expression.this.type.this
|
||||
):
|
||||
return expression.this
|
||||
if (
|
||||
isinstance(expression, (exp.Date, exp.TsOrDsToDate))
|
||||
and expression.this.type
|
||||
and expression.this.type.this == exp.DataType.Type.DATE
|
||||
):
|
||||
return expression.this
|
||||
return expression
|
||||
|
||||
|
||||
|
|
|
@ -19,8 +19,12 @@ def normalize_identifiers(expression: str, dialect: DialectType = None) -> exp.I
|
|||
|
||||
def normalize_identifiers(expression, dialect=None):
|
||||
"""
|
||||
Normalize all unquoted identifiers to either lower or upper case, depending
|
||||
on the dialect. This essentially makes those identifiers case-insensitive.
|
||||
Normalize identifiers by converting them to either lower or upper case,
|
||||
ensuring the semantics are preserved in each case (e.g. by respecting
|
||||
case-sensitivity).
|
||||
|
||||
This transformation reflects how identifiers would be resolved by the engine corresponding
|
||||
to each SQL dialect, and plays a very important role in the standardization of the AST.
|
||||
|
||||
It's possible to make this a no-op by adding a special comment next to the
|
||||
identifier of interest:
|
||||
|
@ -30,7 +34,7 @@ def normalize_identifiers(expression, dialect=None):
|
|||
In this example, the identifier `a` will not be normalized.
|
||||
|
||||
Note:
|
||||
Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even
|
||||
Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even
|
||||
when they're quoted, so in these cases all identifiers are normalized.
|
||||
|
||||
Example:
|
||||
|
|
|
@ -30,6 +30,7 @@ def qualify(
|
|||
validate_qualify_columns: bool = True,
|
||||
quote_identifiers: bool = True,
|
||||
identify: bool = True,
|
||||
infer_csv_schemas: bool = False,
|
||||
) -> exp.Expression:
|
||||
"""
|
||||
Rewrite sqlglot AST to have normalized and qualified tables and columns.
|
||||
|
@ -60,13 +61,21 @@ def qualify(
|
|||
This step is necessary to ensure correctness for case sensitive queries.
|
||||
But this flag is provided in case this step is performed at a later time.
|
||||
identify: If True, quote all identifiers, else only necessary ones.
|
||||
infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.
|
||||
|
||||
Returns:
|
||||
The qualified expression.
|
||||
"""
|
||||
schema = ensure_schema(schema, dialect=dialect)
|
||||
expression = normalize_identifiers(expression, dialect=dialect)
|
||||
expression = qualify_tables(expression, db=db, catalog=catalog, schema=schema, dialect=dialect)
|
||||
expression = qualify_tables(
|
||||
expression,
|
||||
db=db,
|
||||
catalog=catalog,
|
||||
schema=schema,
|
||||
dialect=dialect,
|
||||
infer_csv_schemas=infer_csv_schemas,
|
||||
)
|
||||
|
||||
if isolate_tables:
|
||||
expression = isolate_table_selects(expression, schema=schema)
|
||||
|
|
|
@ -275,6 +275,17 @@ def _expand_alias_refs(scope: Scope, resolver: Resolver, expand_only_groupby: bo
|
|||
if isinstance(projection, exp.Alias):
|
||||
alias_to_expression[projection.alias] = (projection.this, i + 1)
|
||||
|
||||
parent_scope = scope
|
||||
while parent_scope.is_union:
|
||||
parent_scope = parent_scope.parent
|
||||
|
||||
# We shouldn't expand aliases if they match the recursive CTE's columns
|
||||
if parent_scope.is_cte:
|
||||
cte = parent_scope.expression.parent
|
||||
if cte.find_ancestor(exp.With).recursive:
|
||||
for recursive_cte_column in cte.args["alias"].columns or cte.this.selects:
|
||||
alias_to_expression.pop(recursive_cte_column.output_name, None)
|
||||
|
||||
replace_columns(expression.args.get("where"))
|
||||
replace_columns(expression.args.get("group"), literal_index=True)
|
||||
replace_columns(expression.args.get("having"), resolve_table=True)
|
||||
|
|
|
@ -18,6 +18,7 @@ def qualify_tables(
|
|||
db: t.Optional[str | exp.Identifier] = None,
|
||||
catalog: t.Optional[str | exp.Identifier] = None,
|
||||
schema: t.Optional[Schema] = None,
|
||||
infer_csv_schemas: bool = False,
|
||||
dialect: DialectType = None,
|
||||
) -> E:
|
||||
"""
|
||||
|
@ -39,6 +40,7 @@ def qualify_tables(
|
|||
db: Database name
|
||||
catalog: Catalog name
|
||||
schema: A schema to populate
|
||||
infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.
|
||||
dialect: The dialect to parse catalog and schema into.
|
||||
|
||||
Returns:
|
||||
|
@ -102,7 +104,7 @@ def qualify_tables(
|
|||
"alias", exp.TableAlias(this=exp.to_identifier(next_alias_name()))
|
||||
)
|
||||
|
||||
if schema and isinstance(source.this, exp.ReadCSV):
|
||||
if infer_csv_schemas and schema and isinstance(source.this, exp.ReadCSV):
|
||||
with csv_reader(source.this) as reader:
|
||||
header = next(reader)
|
||||
columns = next(reader)
|
||||
|
|
|
@ -65,6 +65,7 @@ class Scope:
|
|||
scope_type=ScopeType.ROOT,
|
||||
lateral_sources=None,
|
||||
cte_sources=None,
|
||||
can_be_correlated=None,
|
||||
):
|
||||
self.expression = expression
|
||||
self.sources = sources or {}
|
||||
|
@ -81,6 +82,7 @@ class Scope:
|
|||
self.cte_scopes = []
|
||||
self.union_scopes = []
|
||||
self.udtf_scopes = []
|
||||
self.can_be_correlated = can_be_correlated
|
||||
self.clear_cache()
|
||||
|
||||
def clear_cache(self):
|
||||
|
@ -110,6 +112,8 @@ class Scope:
|
|||
scope_type=scope_type,
|
||||
cte_sources={**self.cte_sources, **(cte_sources or {})},
|
||||
lateral_sources=lateral_sources.copy() if lateral_sources else None,
|
||||
can_be_correlated=self.can_be_correlated
|
||||
or scope_type in (ScopeType.SUBQUERY, ScopeType.UDTF),
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
@ -261,7 +265,11 @@ class Scope:
|
|||
|
||||
external_columns = [
|
||||
column
|
||||
for scope in itertools.chain(self.subquery_scopes, self.udtf_scopes)
|
||||
for scope in itertools.chain(
|
||||
self.subquery_scopes,
|
||||
self.udtf_scopes,
|
||||
(dts for dts in self.derived_table_scopes if dts.can_be_correlated),
|
||||
)
|
||||
for column in scope.external_columns
|
||||
]
|
||||
|
||||
|
@ -425,10 +433,7 @@ class Scope:
|
|||
@property
|
||||
def is_correlated_subquery(self):
|
||||
"""Determine if this scope is a correlated subquery"""
|
||||
return bool(
|
||||
(self.is_subquery or (self.parent and isinstance(self.parent.expression, exp.Lateral)))
|
||||
and self.external_columns
|
||||
)
|
||||
return bool(self.can_be_correlated and self.external_columns)
|
||||
|
||||
def rename_source(self, old_name, new_name):
|
||||
"""Rename a source in this scope"""
|
||||
|
|
|
@ -117,6 +117,29 @@ def build_pad(args: t.List, is_left: bool = True):
|
|||
)
|
||||
|
||||
|
||||
def build_array_constructor(
|
||||
exp_class: t.Type[E], args: t.List, bracket_kind: TokenType, dialect: Dialect
|
||||
) -> exp.Expression:
|
||||
array_exp = exp_class(expressions=args)
|
||||
|
||||
if exp_class == exp.Array and dialect.HAS_DISTINCT_ARRAY_CONSTRUCTORS:
|
||||
array_exp.set("bracket_notation", bracket_kind == TokenType.L_BRACKET)
|
||||
|
||||
return array_exp
|
||||
|
||||
|
||||
def build_convert_timezone(
|
||||
args: t.List, default_source_tz: t.Optional[str] = None
|
||||
) -> t.Union[exp.ConvertTimezone, exp.Anonymous]:
|
||||
if len(args) == 2:
|
||||
source_tz = exp.Literal.string(default_source_tz) if default_source_tz else None
|
||||
return exp.ConvertTimezone(
|
||||
source_tz=source_tz, target_tz=seq_get(args, 0), timestamp=seq_get(args, 1)
|
||||
)
|
||||
|
||||
return exp.ConvertTimezone.from_arg_list(args)
|
||||
|
||||
|
||||
class _Parser(type):
|
||||
def __new__(cls, clsname, bases, attrs):
|
||||
klass = super().__new__(cls, clsname, bases, attrs)
|
||||
|
@ -144,6 +167,7 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
FUNCTIONS: t.Dict[str, t.Callable] = {
|
||||
**{name: func.from_arg_list for name, func in exp.FUNCTION_BY_NAME.items()},
|
||||
"ARRAY": lambda args, dialect: exp.Array(expressions=args),
|
||||
"CONCAT": lambda args, dialect: exp.Concat(
|
||||
expressions=args,
|
||||
safe=not dialect.STRICT_STRING_CONCAT,
|
||||
|
@ -154,10 +178,16 @@ class Parser(metaclass=_Parser):
|
|||
safe=not dialect.STRICT_STRING_CONCAT,
|
||||
coalesce=dialect.CONCAT_COALESCE,
|
||||
),
|
||||
"CONVERT_TIMEZONE": build_convert_timezone,
|
||||
"DATE_TO_DATE_STR": lambda args: exp.Cast(
|
||||
this=seq_get(args, 0),
|
||||
to=exp.DataType(this=exp.DataType.Type.TEXT),
|
||||
),
|
||||
"GENERATE_DATE_ARRAY": lambda args: exp.GenerateDateArray(
|
||||
start=seq_get(args, 0),
|
||||
end=seq_get(args, 1),
|
||||
step=seq_get(args, 2) or exp.Interval(this=exp.Literal.number(1), unit=exp.var("DAY")),
|
||||
),
|
||||
"GLOB": lambda args: exp.Glob(this=seq_get(args, 1), expression=seq_get(args, 0)),
|
||||
"HEX": build_hex,
|
||||
"JSON_EXTRACT": build_extract_json_with_path(exp.JSONExtract),
|
||||
|
@ -192,6 +222,7 @@ class Parser(metaclass=_Parser):
|
|||
"UNNEST": lambda args: exp.Unnest(expressions=ensure_list(seq_get(args, 0))),
|
||||
"UPPER": build_upper,
|
||||
"VAR_MAP": build_var_map,
|
||||
"COALESCE": lambda args: exp.Coalesce(this=seq_get(args, 0), expressions=args[1:]),
|
||||
}
|
||||
|
||||
NO_PAREN_FUNCTIONS = {
|
||||
|
@ -374,6 +405,11 @@ class Parser(metaclass=_Parser):
|
|||
*DB_CREATABLES,
|
||||
}
|
||||
|
||||
ALTERABLES = {
|
||||
TokenType.TABLE,
|
||||
TokenType.VIEW,
|
||||
}
|
||||
|
||||
# Tokens that can represent identifiers
|
||||
ID_VAR_TOKENS = {
|
||||
TokenType.ALL,
|
||||
|
@ -433,6 +469,7 @@ class Parser(metaclass=_Parser):
|
|||
TokenType.RECURSIVE,
|
||||
TokenType.REFERENCES,
|
||||
TokenType.REFRESH,
|
||||
TokenType.RENAME,
|
||||
TokenType.REPLACE,
|
||||
TokenType.RIGHT,
|
||||
TokenType.ROLLUP,
|
||||
|
@ -842,6 +879,7 @@ class Parser(metaclass=_Parser):
|
|||
"DYNAMIC": lambda self: self.expression(exp.DynamicProperty),
|
||||
"DISTKEY": lambda self: self._parse_distkey(),
|
||||
"DISTSTYLE": lambda self: self._parse_property_assignment(exp.DistStyleProperty),
|
||||
"EMPTY": lambda self: self.expression(exp.EmptyProperty),
|
||||
"ENGINE": lambda self: self._parse_property_assignment(exp.EngineProperty),
|
||||
"EXECUTE": lambda self: self._parse_property_assignment(exp.ExecuteAsProperty),
|
||||
"EXTERNAL": lambda self: self.expression(exp.ExternalProperty),
|
||||
|
@ -885,6 +923,7 @@ class Parser(metaclass=_Parser):
|
|||
"REMOTE": lambda self: self._parse_remote_with_connection(),
|
||||
"RETURNS": lambda self: self._parse_returns(),
|
||||
"STRICT": lambda self: self.expression(exp.StrictProperty),
|
||||
"STREAMING": lambda self: self.expression(exp.StreamingTableProperty),
|
||||
"ROW": lambda self: self._parse_row(),
|
||||
"ROW_FORMAT": lambda self: self._parse_property_assignment(exp.RowFormatProperty),
|
||||
"SAMPLE": lambda self: self.expression(
|
||||
|
@ -892,9 +931,7 @@ class Parser(metaclass=_Parser):
|
|||
),
|
||||
"SECURE": lambda self: self.expression(exp.SecureProperty),
|
||||
"SET": lambda self: self.expression(exp.SetProperty, multi=False),
|
||||
"SETTINGS": lambda self: self.expression(
|
||||
exp.SettingsProperty, expressions=self._parse_csv(self._parse_set_item)
|
||||
),
|
||||
"SETTINGS": lambda self: self._parse_settings_property(),
|
||||
"SHARING": lambda self: self._parse_property_assignment(exp.SharingProperty),
|
||||
"SORTKEY": lambda self: self._parse_sortkey(),
|
||||
"SOURCE": lambda self: self._parse_dict_property(this="SOURCE"),
|
||||
|
@ -992,6 +1029,7 @@ class Parser(metaclass=_Parser):
|
|||
"DROP": lambda self: self._parse_alter_table_drop(),
|
||||
"RENAME": lambda self: self._parse_alter_table_rename(),
|
||||
"SET": lambda self: self._parse_alter_table_set(),
|
||||
"AS": lambda self: self._parse_select(),
|
||||
}
|
||||
|
||||
ALTER_ALTER_PARSERS = {
|
||||
|
@ -1628,7 +1666,7 @@ class Parser(metaclass=_Parser):
|
|||
temporary = self._match(TokenType.TEMPORARY)
|
||||
materialized = self._match_text_seq("MATERIALIZED")
|
||||
|
||||
kind = self._match_set(self.CREATABLES) and self._prev.text
|
||||
kind = self._match_set(self.CREATABLES) and self._prev.text.upper()
|
||||
if not kind:
|
||||
return self._parse_as_command(start)
|
||||
|
||||
|
@ -1650,7 +1688,7 @@ class Parser(metaclass=_Parser):
|
|||
exists=if_exists,
|
||||
this=table,
|
||||
expressions=expressions,
|
||||
kind=kind.upper(),
|
||||
kind=self.dialect.CREATABLE_KIND_MAPPING.get(kind) or kind,
|
||||
temporary=temporary,
|
||||
materialized=materialized,
|
||||
cascade=self._match_text_seq("CASCADE"),
|
||||
|
@ -1676,6 +1714,7 @@ class Parser(metaclass=_Parser):
|
|||
or self._match_pair(TokenType.OR, TokenType.REPLACE)
|
||||
or self._match_pair(TokenType.OR, TokenType.ALTER)
|
||||
)
|
||||
refresh = self._match_pair(TokenType.OR, TokenType.REFRESH)
|
||||
|
||||
unique = self._match(TokenType.UNIQUE)
|
||||
|
||||
|
@ -1792,7 +1831,6 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
# exp.Properties.Location.POST_INDEX
|
||||
extend_props(self._parse_properties())
|
||||
|
||||
if not index:
|
||||
break
|
||||
else:
|
||||
|
@ -1813,12 +1851,14 @@ class Parser(metaclass=_Parser):
|
|||
if self._curr and not self._match_set((TokenType.R_PAREN, TokenType.COMMA), advance=False):
|
||||
return self._parse_as_command(start)
|
||||
|
||||
create_kind_text = create_token.text.upper()
|
||||
return self.expression(
|
||||
exp.Create,
|
||||
comments=comments,
|
||||
this=this,
|
||||
kind=create_token.text.upper(),
|
||||
kind=self.dialect.CREATABLE_KIND_MAPPING.get(create_kind_text) or create_kind_text,
|
||||
replace=replace,
|
||||
refresh=refresh,
|
||||
unique=unique,
|
||||
expression=expression,
|
||||
exists=exists,
|
||||
|
@ -1979,6 +2019,11 @@ class Parser(metaclass=_Parser):
|
|||
exp.FallbackProperty, no=no, protection=self._match_text_seq("PROTECTION")
|
||||
)
|
||||
|
||||
def _parse_settings_property(self) -> exp.SettingsProperty:
|
||||
return self.expression(
|
||||
exp.SettingsProperty, expressions=self._parse_csv(self._parse_assignment)
|
||||
)
|
||||
|
||||
def _parse_volatile_property(self) -> exp.VolatileProperty | exp.StabilityProperty:
|
||||
if self._index >= 2:
|
||||
pre_volatile_token = self._tokens[self._index - 2]
|
||||
|
@ -2451,8 +2496,14 @@ class Parser(metaclass=_Parser):
|
|||
this = self._parse_table(schema=True)
|
||||
properties = self._parse_properties()
|
||||
expressions = properties.expressions if properties else None
|
||||
partition = self._parse_partition()
|
||||
return self.expression(
|
||||
exp.Describe, this=this, style=style, kind=kind, expressions=expressions
|
||||
exp.Describe,
|
||||
this=this,
|
||||
style=style,
|
||||
kind=kind,
|
||||
expressions=expressions,
|
||||
partition=partition,
|
||||
)
|
||||
|
||||
def _parse_insert(self) -> exp.Insert:
|
||||
|
@ -2498,6 +2549,8 @@ class Parser(metaclass=_Parser):
|
|||
by_name=self._match_text_seq("BY", "NAME"),
|
||||
exists=self._parse_exists(),
|
||||
where=self._match_pair(TokenType.REPLACE, TokenType.WHERE) and self._parse_assignment(),
|
||||
partition=self._match(TokenType.PARTITION_BY) and self._parse_partitioned_by(),
|
||||
settings=self._match_text_seq("SETTINGS") and self._parse_settings_property(),
|
||||
expression=self._parse_derived_table_values() or self._parse_ddl_select(),
|
||||
conflict=self._parse_on_conflict(),
|
||||
returning=returning or self._parse_returning(),
|
||||
|
@ -2830,6 +2883,10 @@ class Parser(metaclass=_Parser):
|
|||
table = self._match(TokenType.TABLE)
|
||||
this = self._parse_select() or self._parse_string() or self._parse_table()
|
||||
return self.expression(exp.Summarize, this=this, table=table)
|
||||
elif self._match(TokenType.DESCRIBE):
|
||||
this = self._parse_describe()
|
||||
elif self._match_text_seq("STREAM"):
|
||||
this = self.expression(exp.Stream, this=self._parse_function())
|
||||
else:
|
||||
this = None
|
||||
|
||||
|
@ -3173,6 +3230,15 @@ class Parser(metaclass=_Parser):
|
|||
self._match_set(self.JOIN_KINDS) and self._prev,
|
||||
)
|
||||
|
||||
def _parse_using_identifiers(self) -> t.List[exp.Expression]:
|
||||
def _parse_column_as_identifier() -> t.Optional[exp.Expression]:
|
||||
this = self._parse_column()
|
||||
if isinstance(this, exp.Column):
|
||||
return this.this
|
||||
return this
|
||||
|
||||
return self._parse_wrapped_csv(_parse_column_as_identifier, optional=True)
|
||||
|
||||
def _parse_join(
|
||||
self, skip_join_token: bool = False, parse_bracket: bool = False
|
||||
) -> t.Optional[exp.Join]:
|
||||
|
@ -3213,9 +3279,11 @@ class Parser(metaclass=_Parser):
|
|||
if self._match(TokenType.ON):
|
||||
kwargs["on"] = self._parse_assignment()
|
||||
elif self._match(TokenType.USING):
|
||||
kwargs["using"] = self._parse_wrapped_id_vars()
|
||||
elif not isinstance(kwargs["this"], exp.Unnest) and not (
|
||||
kind and kind.token_type == TokenType.CROSS
|
||||
kwargs["using"] = self._parse_using_identifiers()
|
||||
elif (
|
||||
not (outer_apply or cross_apply)
|
||||
and not isinstance(kwargs["this"], exp.Unnest)
|
||||
and not (kind and kind.token_type == TokenType.CROSS)
|
||||
):
|
||||
index = self._index
|
||||
joins: t.Optional[list] = list(self._parse_joins())
|
||||
|
@ -3223,7 +3291,7 @@ class Parser(metaclass=_Parser):
|
|||
if joins and self._match(TokenType.ON):
|
||||
kwargs["on"] = self._parse_assignment()
|
||||
elif joins and self._match(TokenType.USING):
|
||||
kwargs["using"] = self._parse_wrapped_id_vars()
|
||||
kwargs["using"] = self._parse_using_identifiers()
|
||||
else:
|
||||
joins = None
|
||||
self._retreat(index)
|
||||
|
@ -3607,7 +3675,10 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
def _parse_derived_table_values(self) -> t.Optional[exp.Values]:
|
||||
is_derived = self._match_pair(TokenType.L_PAREN, TokenType.VALUES)
|
||||
if not is_derived and not self._match_text_seq("VALUES"):
|
||||
if not is_derived and not (
|
||||
# ClickHouse's `FORMAT Values` is equivalent to `VALUES`
|
||||
self._match_text_seq("VALUES") or self._match_text_seq("FORMAT", "VALUES")
|
||||
):
|
||||
return None
|
||||
|
||||
expressions = self._parse_csv(self._parse_value)
|
||||
|
@ -3707,13 +3778,15 @@ class Parser(metaclass=_Parser):
|
|||
exp.Pivot, this=this, expressions=expressions, using=using, group=group
|
||||
)
|
||||
|
||||
def _parse_pivot_in(self) -> exp.In:
|
||||
def _parse_pivot_in(self) -> exp.In | exp.PivotAny:
|
||||
def _parse_aliased_expression() -> t.Optional[exp.Expression]:
|
||||
this = self._parse_assignment()
|
||||
this = self._parse_select_or_expression()
|
||||
|
||||
self._match(TokenType.ALIAS)
|
||||
alias = self._parse_field()
|
||||
alias = self._parse_bitwise()
|
||||
if alias:
|
||||
if isinstance(alias, exp.Column) and not alias.db:
|
||||
alias = alias.this
|
||||
return self.expression(exp.PivotAlias, this=this, alias=alias)
|
||||
|
||||
return this
|
||||
|
@ -3723,10 +3796,14 @@ class Parser(metaclass=_Parser):
|
|||
if not self._match_pair(TokenType.IN, TokenType.L_PAREN):
|
||||
self.raise_error("Expecting IN (")
|
||||
|
||||
if self._match(TokenType.ANY):
|
||||
expr: exp.PivotAny | exp.In = self.expression(exp.PivotAny, this=self._parse_order())
|
||||
else:
|
||||
aliased_expressions = self._parse_csv(_parse_aliased_expression)
|
||||
expr = self.expression(exp.In, this=value, expressions=aliased_expressions)
|
||||
|
||||
self._match_r_paren()
|
||||
return self.expression(exp.In, this=value, expressions=aliased_expressions)
|
||||
return expr
|
||||
|
||||
def _parse_pivot(self) -> t.Optional[exp.Pivot]:
|
||||
index = self._index
|
||||
|
@ -3763,6 +3840,9 @@ class Parser(metaclass=_Parser):
|
|||
self.raise_error("Expecting FOR")
|
||||
|
||||
field = self._parse_pivot_in()
|
||||
default_on_null = self._match_text_seq("DEFAULT", "ON", "NULL") and self._parse_wrapped(
|
||||
self._parse_bitwise
|
||||
)
|
||||
|
||||
self._match_r_paren()
|
||||
|
||||
|
@ -3772,6 +3852,7 @@ class Parser(metaclass=_Parser):
|
|||
field=field,
|
||||
unpivot=unpivot,
|
||||
include_nulls=include_nulls,
|
||||
default_on_null=default_on_null,
|
||||
)
|
||||
|
||||
if not self._match_set((TokenType.PIVOT, TokenType.UNPIVOT), advance=False):
|
||||
|
@ -3934,7 +4015,6 @@ class Parser(metaclass=_Parser):
|
|||
exp.Order,
|
||||
this=this,
|
||||
expressions=self._parse_csv(self._parse_ordered),
|
||||
interpolate=self._parse_interpolate(),
|
||||
siblings=siblings,
|
||||
)
|
||||
|
||||
|
@ -3979,6 +4059,7 @@ class Parser(metaclass=_Parser):
|
|||
"from": self._match(TokenType.FROM) and self._parse_bitwise(),
|
||||
"to": self._match_text_seq("TO") and self._parse_bitwise(),
|
||||
"step": self._match_text_seq("STEP") and self._parse_bitwise(),
|
||||
"interpolate": self._parse_interpolate(),
|
||||
},
|
||||
)
|
||||
else:
|
||||
|
@ -4132,6 +4213,11 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
def _parse_assignment(self) -> t.Optional[exp.Expression]:
|
||||
this = self._parse_disjunction()
|
||||
if not this and self._next and self._next.token_type in self.ASSIGNMENT:
|
||||
# This allows us to parse <non-identifier token> := <expr>
|
||||
this = exp.column(
|
||||
t.cast(str, self._advance_any(ignore_reserved=True) and self._prev.text)
|
||||
)
|
||||
|
||||
while self._match_set(self.ASSIGNMENT):
|
||||
this = self.expression(
|
||||
|
@ -4175,13 +4261,19 @@ class Parser(metaclass=_Parser):
|
|||
this = self.expression(exp.Not, this=this)
|
||||
|
||||
if negate:
|
||||
this = self.expression(exp.Not, this=this)
|
||||
this = self._negate_range(this)
|
||||
|
||||
if self._match(TokenType.IS):
|
||||
this = self._parse_is(this)
|
||||
|
||||
return this
|
||||
|
||||
def _negate_range(self, this: t.Optional[exp.Expression] = None) -> t.Optional[exp.Expression]:
|
||||
if not this:
|
||||
return this
|
||||
|
||||
return self.expression(exp.Not, this=this)
|
||||
|
||||
def _parse_is(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
|
||||
index = self._index - 1
|
||||
negate = self._match(TokenType.NOT)
|
||||
|
@ -4322,7 +4414,26 @@ class Parser(metaclass=_Parser):
|
|||
return this
|
||||
|
||||
def _parse_term(self) -> t.Optional[exp.Expression]:
|
||||
return self._parse_tokens(self._parse_factor, self.TERM)
|
||||
this = self._parse_factor()
|
||||
|
||||
while self._match_set(self.TERM):
|
||||
klass = self.TERM[self._prev.token_type]
|
||||
comments = self._prev_comments
|
||||
expression = self._parse_factor()
|
||||
|
||||
this = self.expression(klass, this=this, comments=comments, expression=expression)
|
||||
|
||||
if isinstance(this, exp.Collate):
|
||||
expr = this.expression
|
||||
|
||||
# Preserve collations such as pg_catalog."default" (Postgres) as columns, otherwise
|
||||
# fallback to Identifier / Var
|
||||
if isinstance(expr, exp.Column) and len(expr.parts) == 1:
|
||||
ident = expr.this
|
||||
if isinstance(ident, exp.Identifier):
|
||||
this.set("expression", ident if ident.quoted else exp.var(ident.name))
|
||||
|
||||
return this
|
||||
|
||||
def _parse_factor(self) -> t.Optional[exp.Expression]:
|
||||
parse_method = self._parse_exponent if self.EXPONENT else self._parse_unary
|
||||
|
@ -4610,6 +4721,7 @@ class Parser(metaclass=_Parser):
|
|||
matched_array = self._match(TokenType.ARRAY)
|
||||
|
||||
while self._curr:
|
||||
datatype_token = self._prev.token_type
|
||||
matched_l_bracket = self._match(TokenType.L_BRACKET)
|
||||
if not matched_l_bracket and not matched_array:
|
||||
break
|
||||
|
@ -4619,8 +4731,12 @@ class Parser(metaclass=_Parser):
|
|||
if (
|
||||
values
|
||||
and not schema
|
||||
and this.is_type(exp.DataType.Type.ARRAY, exp.DataType.Type.MAP)
|
||||
and (
|
||||
not self.dialect.SUPPORTS_FIXED_SIZE_ARRAYS or datatype_token == TokenType.ARRAY
|
||||
)
|
||||
):
|
||||
# Retreating here means that we should not parse the following values as part of the data type, e.g. in DuckDB
|
||||
# ARRAY[1] should retreat and instead be parsed into exp.Array in contrast to INT[x][y] which denotes a fixed-size array data type
|
||||
self._retreat(index)
|
||||
break
|
||||
|
||||
|
@ -5407,11 +5523,18 @@ class Parser(metaclass=_Parser):
|
|||
if bracket_kind == TokenType.L_BRACE:
|
||||
this = self.expression(exp.Struct, expressions=self._kv_to_prop_eq(expressions))
|
||||
elif not this:
|
||||
this = self.expression(exp.Array, expressions=expressions)
|
||||
this = build_array_constructor(
|
||||
exp.Array, args=expressions, bracket_kind=bracket_kind, dialect=self.dialect
|
||||
)
|
||||
else:
|
||||
constructor_type = self.ARRAY_CONSTRUCTORS.get(this.name.upper())
|
||||
if constructor_type:
|
||||
return self.expression(constructor_type, expressions=expressions)
|
||||
return build_array_constructor(
|
||||
constructor_type,
|
||||
args=expressions,
|
||||
bracket_kind=bracket_kind,
|
||||
dialect=self.dialect,
|
||||
)
|
||||
|
||||
expressions = apply_index_offset(this, expressions, -self.dialect.INDEX_OFFSET)
|
||||
this = self.expression(exp.Bracket, this=this, expressions=expressions)
|
||||
|
@ -6440,10 +6563,11 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
return alter_set
|
||||
|
||||
def _parse_alter(self) -> exp.AlterTable | exp.Command:
|
||||
def _parse_alter(self) -> exp.Alter | exp.Command:
|
||||
start = self._prev
|
||||
|
||||
if not self._match(TokenType.TABLE):
|
||||
alter_token = self._match_set(self.ALTERABLES) and self._prev
|
||||
if not alter_token:
|
||||
return self._parse_as_command(start)
|
||||
|
||||
exists = self._parse_exists()
|
||||
|
@ -6461,8 +6585,9 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
if not self._curr and actions:
|
||||
return self.expression(
|
||||
exp.AlterTable,
|
||||
exp.Alter,
|
||||
this=this,
|
||||
kind=alter_token.text.upper(),
|
||||
exists=exists,
|
||||
actions=actions,
|
||||
only=only,
|
||||
|
|
|
@ -340,6 +340,7 @@ class TokenType(AutoName):
|
|||
RANGE = auto()
|
||||
RECURSIVE = auto()
|
||||
REFRESH = auto()
|
||||
RENAME = auto()
|
||||
REPLACE = auto()
|
||||
RETURNING = auto()
|
||||
REFERENCES = auto()
|
||||
|
@ -529,6 +530,7 @@ class _Tokenizer(type):
|
|||
},
|
||||
heredoc_tag_is_identifier=klass.HEREDOC_TAG_IS_IDENTIFIER,
|
||||
string_escapes_allowed_in_raw_strings=klass.STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS,
|
||||
nested_comments=klass.NESTED_COMMENTS,
|
||||
)
|
||||
token_types = RsTokenTypeSettings(
|
||||
bit_string=_TOKEN_TYPE_TO_INDEX[TokenType.BIT_STRING],
|
||||
|
@ -608,6 +610,8 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
# Whether string escape characters function as such when placed within raw strings
|
||||
STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS = True
|
||||
|
||||
NESTED_COMMENTS = True
|
||||
|
||||
# Autofilled
|
||||
_COMMENTS: t.Dict[str, str] = {}
|
||||
_FORMAT_STRINGS: t.Dict[str, t.Tuple[str, TokenType]] = {}
|
||||
|
@ -753,6 +757,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
"RANGE": TokenType.RANGE,
|
||||
"RECURSIVE": TokenType.RECURSIVE,
|
||||
"REGEXP": TokenType.RLIKE,
|
||||
"RENAME": TokenType.RENAME,
|
||||
"REPLACE": TokenType.REPLACE,
|
||||
"RETURNING": TokenType.RETURNING,
|
||||
"REFERENCES": TokenType.REFERENCES,
|
||||
|
@ -913,6 +918,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
TokenType.EXECUTE,
|
||||
TokenType.FETCH,
|
||||
TokenType.SHOW,
|
||||
TokenType.RENAME,
|
||||
}
|
||||
|
||||
COMMAND_PREFIX_TOKENS = {TokenType.SEMICOLON, TokenType.BEGIN}
|
||||
|
@ -1181,7 +1187,11 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
self._advance(alnum=True)
|
||||
|
||||
# Nested comments are allowed by some dialects, e.g. databricks, duckdb, postgres
|
||||
if not self._end and self._chars(comment_end_size) == comment_start:
|
||||
if (
|
||||
self.NESTED_COMMENTS
|
||||
and not self._end
|
||||
and self._chars(comment_end_size) == comment_start
|
||||
):
|
||||
self._advance(comment_start_size)
|
||||
comment_count += 1
|
||||
|
||||
|
|
|
@ -55,6 +55,76 @@ def preprocess(
|
|||
return _to_sql
|
||||
|
||||
|
||||
def unnest_generate_date_array_using_recursive_cte(expression: exp.Expression) -> exp.Expression:
|
||||
if isinstance(expression, exp.Select):
|
||||
count = 0
|
||||
recursive_ctes = []
|
||||
|
||||
for unnest in expression.find_all(exp.Unnest):
|
||||
if (
|
||||
not isinstance(unnest.parent, (exp.From, exp.Join))
|
||||
or len(unnest.expressions) != 1
|
||||
or not isinstance(unnest.expressions[0], exp.GenerateDateArray)
|
||||
):
|
||||
continue
|
||||
|
||||
generate_date_array = unnest.expressions[0]
|
||||
start = generate_date_array.args.get("start")
|
||||
end = generate_date_array.args.get("end")
|
||||
step = generate_date_array.args.get("step")
|
||||
|
||||
if not start or not end or not isinstance(step, exp.Interval):
|
||||
continue
|
||||
|
||||
alias = unnest.args.get("alias")
|
||||
column_name = alias.columns[0] if isinstance(alias, exp.TableAlias) else "date_value"
|
||||
|
||||
start = exp.cast(start, "date")
|
||||
date_add = exp.func(
|
||||
"date_add", column_name, exp.Literal.number(step.name), step.args.get("unit")
|
||||
)
|
||||
cast_date_add = exp.cast(date_add, "date")
|
||||
|
||||
cte_name = "_generated_dates" + (f"_{count}" if count else "")
|
||||
|
||||
base_query = exp.select(start.as_(column_name))
|
||||
recursive_query = (
|
||||
exp.select(cast_date_add)
|
||||
.from_(cte_name)
|
||||
.where(cast_date_add <= exp.cast(end, "date"))
|
||||
)
|
||||
cte_query = base_query.union(recursive_query, distinct=False)
|
||||
|
||||
generate_dates_query = exp.select(column_name).from_(cte_name)
|
||||
unnest.replace(generate_dates_query.subquery(cte_name))
|
||||
|
||||
recursive_ctes.append(
|
||||
exp.alias_(exp.CTE(this=cte_query), cte_name, table=[column_name])
|
||||
)
|
||||
count += 1
|
||||
|
||||
if recursive_ctes:
|
||||
with_expression = expression.args.get("with") or exp.With()
|
||||
with_expression.set("recursive", True)
|
||||
with_expression.set("expressions", [*recursive_ctes, *with_expression.expressions])
|
||||
expression.set("with", with_expression)
|
||||
|
||||
return expression
|
||||
|
||||
|
||||
def unnest_generate_series(expression: exp.Expression) -> exp.Expression:
|
||||
"""Unnests GENERATE_SERIES or SEQUENCE table references."""
|
||||
this = expression.this
|
||||
if isinstance(expression, exp.Table) and isinstance(this, exp.GenerateSeries):
|
||||
unnest = exp.Unnest(expressions=[this])
|
||||
if expression.alias:
|
||||
return exp.alias_(unnest, alias="_u", table=[expression.alias], copy=False)
|
||||
|
||||
return unnest
|
||||
|
||||
return expression
|
||||
|
||||
|
||||
def unalias_group(expression: exp.Expression) -> exp.Expression:
|
||||
"""
|
||||
Replace references to select aliases in GROUP BY clauses.
|
||||
|
|
2
sqlglotrs/Cargo.lock
generated
2
sqlglotrs/Cargo.lock
generated
|
@ -188,7 +188,7 @@ checksum = "4dccd0940a2dcdf68d092b8cbab7dc0ad8fa938bf95787e1b916b0e3d0e8e970"
|
|||
|
||||
[[package]]
|
||||
name = "sqlglotrs"
|
||||
version = "0.2.8"
|
||||
version = "0.2.9"
|
||||
dependencies = [
|
||||
"pyo3",
|
||||
]
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
[package]
|
||||
name = "sqlglotrs"
|
||||
version = "0.2.8"
|
||||
version = "0.2.9"
|
||||
edition = "2021"
|
||||
|
||||
[lib]
|
||||
|
|
|
@ -77,6 +77,7 @@ pub struct TokenizerSettings {
|
|||
pub command_prefix_tokens: HashSet<TokenType>,
|
||||
pub heredoc_tag_is_identifier: bool,
|
||||
pub string_escapes_allowed_in_raw_strings: bool,
|
||||
pub nested_comments: bool,
|
||||
}
|
||||
|
||||
#[pymethods]
|
||||
|
@ -100,6 +101,7 @@ impl TokenizerSettings {
|
|||
command_prefix_tokens: HashSet<TokenType>,
|
||||
heredoc_tag_is_identifier: bool,
|
||||
string_escapes_allowed_in_raw_strings: bool,
|
||||
nested_comments: bool,
|
||||
) -> Self {
|
||||
let to_char = |v: &String| {
|
||||
if v.len() == 1 {
|
||||
|
@ -150,6 +152,7 @@ impl TokenizerSettings {
|
|||
command_prefix_tokens,
|
||||
heredoc_tag_is_identifier,
|
||||
string_escapes_allowed_in_raw_strings,
|
||||
nested_comments,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
@ -375,7 +375,7 @@ impl<'a> TokenizerState<'a> {
|
|||
self.advance(1)?;
|
||||
|
||||
// Nested comments are allowed by some dialects, e.g. databricks, duckdb, postgres
|
||||
if !self.is_end && self.chars(comment_start_size) == *comment_start {
|
||||
if self.settings.nested_comments && !self.is_end && self.chars(comment_start_size) == *comment_start {
|
||||
self.advance(comment_start_size as isize)?;
|
||||
comment_count += 1
|
||||
}
|
||||
|
|
|
@ -6,6 +6,15 @@ class TestAthena(Validator):
|
|||
maxDiff = None
|
||||
|
||||
def test_athena(self):
|
||||
self.validate_identity(
|
||||
"CREATE TABLE IF NOT EXISTS t (name STRING) LOCATION 's3://bucket/tmp/mytable/' TBLPROPERTIES ('table_type'='iceberg', 'FORMAT'='parquet')"
|
||||
)
|
||||
self.validate_identity(
|
||||
"UNLOAD (SELECT name1, address1, comment1, key1 FROM table1) "
|
||||
"TO 's3://amzn-s3-demo-bucket/ partitioned/' "
|
||||
"WITH (format = 'TEXTFILE', partitioned_by = ARRAY['key1'])",
|
||||
check_command_warning=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"""USING EXTERNAL FUNCTION some_function(input VARBINARY)
|
||||
RETURNS VARCHAR
|
||||
|
@ -14,7 +23,3 @@ class TestAthena(Validator):
|
|||
some_function(1)""",
|
||||
check_command_warning=True,
|
||||
)
|
||||
|
||||
self.validate_identity(
|
||||
"CREATE TABLE IF NOT EXISTS t (name STRING) LOCATION 's3://bucket/tmp/mytable/' TBLPROPERTIES ('table_type'='iceberg', 'FORMAT'='parquet')"
|
||||
)
|
||||
|
|
|
@ -1,4 +1,6 @@
|
|||
from unittest import mock
|
||||
import datetime
|
||||
import pytz
|
||||
|
||||
from sqlglot import (
|
||||
ErrorLevel,
|
||||
|
@ -103,6 +105,7 @@ LANGUAGE js AS
|
|||
select_with_quoted_udf = self.validate_identity("SELECT `p.d.UdF`(data) FROM `p.d.t`")
|
||||
self.assertEqual(select_with_quoted_udf.selects[0].name, "p.d.UdF")
|
||||
|
||||
self.validate_identity("SELECT * FROM READ_CSV('bla.csv')")
|
||||
self.validate_identity("CAST(x AS STRUCT<list ARRAY<INT64>>)")
|
||||
self.validate_identity("assert.true(1 = 1)")
|
||||
self.validate_identity("SELECT ARRAY_TO_STRING(list, '--') AS text")
|
||||
|
@ -446,7 +449,7 @@ LANGUAGE js AS
|
|||
write={
|
||||
"bigquery": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE), MONTH)",
|
||||
"duckdb": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
|
||||
"clickhouse": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
|
||||
"clickhouse": "SELECT LAST_DAY(CAST('2008-11-25' AS Nullable(DATE)))",
|
||||
"mysql": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
|
||||
"oracle": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
|
||||
"postgres": "SELECT CAST(DATE_TRUNC('MONTH', CAST('2008-11-25' AS DATE)) + INTERVAL '1 MONTH' - INTERVAL '1 DAY' AS DATE)",
|
||||
|
@ -510,6 +513,20 @@ LANGUAGE js AS
|
|||
"duckdb": "SELECT STRFTIME(CAST('2023-12-25' AS DATE), '%Y%m%d')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT FORMAT_DATETIME('%Y%m%d %H:%M:%S', DATETIME '2023-12-25 15:30:00')",
|
||||
write={
|
||||
"bigquery": "SELECT FORMAT_DATETIME('%Y%m%d %H:%M:%S', CAST('2023-12-25 15:30:00' AS DATETIME))",
|
||||
"duckdb": "SELECT STRFTIME(CAST('2023-12-25 15:30:00' AS TIMESTAMP), '%Y%m%d %H:%M:%S')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT FORMAT_DATETIME('%x', '2023-12-25 15:30:00')",
|
||||
write={
|
||||
"bigquery": "SELECT FORMAT_DATETIME('%x', '2023-12-25 15:30:00')",
|
||||
"duckdb": "SELECT STRFTIME(CAST('2023-12-25 15:30:00' AS TIMESTAMP), '%x')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT COUNTIF(x)",
|
||||
read={
|
||||
|
@ -636,6 +653,7 @@ LANGUAGE js AS
|
|||
write={
|
||||
"bigquery": "SELECT DATETIME_TRUNC('2023-01-01T01:01:01', HOUR)",
|
||||
"databricks": "SELECT DATE_TRUNC('HOUR', '2023-01-01T01:01:01')",
|
||||
"duckdb": "SELECT DATE_TRUNC('HOUR', CAST('2023-01-01T01:01:01' AS DATETIME))",
|
||||
},
|
||||
),
|
||||
)
|
||||
|
@ -1209,10 +1227,9 @@ LANGUAGE js AS
|
|||
"SELECT * FROM a WHERE b IN UNNEST([1, 2, 3])",
|
||||
write={
|
||||
"bigquery": "SELECT * FROM a WHERE b IN UNNEST([1, 2, 3])",
|
||||
"mysql": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY(1, 2, 3)))",
|
||||
"presto": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY[1, 2, 3]))",
|
||||
"hive": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY(1, 2, 3)))",
|
||||
"spark": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY(1, 2, 3)))",
|
||||
"hive": "SELECT * FROM a WHERE b IN (SELECT EXPLODE(ARRAY(1, 2, 3)))",
|
||||
"spark": "SELECT * FROM a WHERE b IN (SELECT EXPLODE(ARRAY(1, 2, 3)))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
@ -1256,6 +1273,13 @@ LANGUAGE js AS
|
|||
"starrocks": "DATE_DIFF('MINUTE', CAST('2010-07-07' AS DATE), CAST('2008-12-25' AS DATE))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"DATE_DIFF('2021-01-01', '2020-01-01', DAY)",
|
||||
write={
|
||||
"bigquery": "DATE_DIFF('2021-01-01', '2020-01-01', DAY)",
|
||||
"duckdb": "DATE_DIFF('DAY', CAST('2020-01-01' AS DATE), CAST('2021-01-01' AS DATE))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CURRENT_DATE('UTC')",
|
||||
write={
|
||||
|
@ -1402,6 +1426,57 @@ WHERE
|
|||
"": "SELECT LENGTH(foo)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT TIME_DIFF('12:00:00', '12:30:00', MINUTE)",
|
||||
write={
|
||||
"duckdb": "SELECT DATE_DIFF('MINUTE', CAST('12:30:00' AS TIME), CAST('12:00:00' AS TIME))",
|
||||
"bigquery": "SELECT TIME_DIFF('12:00:00', '12:30:00', MINUTE)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"ARRAY_CONCAT([1, 2], [3, 4], [5, 6])",
|
||||
write={
|
||||
"bigquery": "ARRAY_CONCAT([1, 2], [3, 4], [5, 6])",
|
||||
"duckdb": "ARRAY_CONCAT([1, 2], ARRAY_CONCAT([3, 4], [5, 6]))",
|
||||
"postgres": "ARRAY_CAT(ARRAY[1, 2], ARRAY_CAT(ARRAY[3, 4], ARRAY[5, 6]))",
|
||||
"redshift": "ARRAY_CONCAT(ARRAY(1, 2), ARRAY_CONCAT(ARRAY(3, 4), ARRAY(5, 6)))",
|
||||
"snowflake": "ARRAY_CAT([1, 2], ARRAY_CAT([3, 4], [5, 6]))",
|
||||
"hive": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
|
||||
"spark2": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
|
||||
"spark": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
|
||||
"databricks": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
|
||||
"presto": "CONCAT(ARRAY[1, 2], ARRAY[3, 4], ARRAY[5, 6])",
|
||||
"trino": "CONCAT(ARRAY[1, 2], ARRAY[3, 4], ARRAY[5, 6])",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08')",
|
||||
write={
|
||||
"duckdb": "SELECT CAST(GENERATE_SERIES(CAST('2016-10-05' AS DATE), CAST('2016-10-08' AS DATE), INTERVAL 1 DAY) AS DATE[])",
|
||||
"bigquery": "SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08', INTERVAL 1 DAY)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08', INTERVAL '1' MONTH)",
|
||||
write={
|
||||
"duckdb": "SELECT CAST(GENERATE_SERIES(CAST('2016-10-05' AS DATE), CAST('2016-10-08' AS DATE), INTERVAL '1' MONTH) AS DATE[])",
|
||||
"bigquery": "SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08', INTERVAL '1' MONTH)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-07 00:00:00', INTERVAL '1' DAY)",
|
||||
write={
|
||||
"duckdb": "SELECT GENERATE_SERIES(CAST('2016-10-05 00:00:00' AS TIMESTAMP), CAST('2016-10-07 00:00:00' AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"bigquery": "SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-07 00:00:00', INTERVAL '1' DAY)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT PARSE_DATE('%A %b %e %Y', 'Thursday Dec 25 2008')",
|
||||
write={
|
||||
"bigquery": "SELECT PARSE_DATE('%A %b %e %Y', 'Thursday Dec 25 2008')",
|
||||
"duckdb": "SELECT CAST(STRPTIME('Thursday Dec 25 2008', '%A %b %-d %Y') AS DATE)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_errors(self):
|
||||
with self.assertRaises(TokenError):
|
||||
|
@ -1794,14 +1869,14 @@ OPTIONS (
|
|||
"SELECT * FROM UNNEST(ARRAY<STRUCT<x INT64>>[])",
|
||||
write={
|
||||
"bigquery": "SELECT * FROM UNNEST(CAST([] AS ARRAY<STRUCT<x INT64>>))",
|
||||
"duckdb": "SELECT * FROM UNNEST(CAST([] AS STRUCT(x BIGINT)[]))",
|
||||
"duckdb": "SELECT * FROM (SELECT UNNEST(CAST([] AS STRUCT(x BIGINT)[]), max_depth => 2))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM UNNEST(ARRAY<STRUCT<device_id INT64, time DATETIME, signal INT64, state STRING>>[STRUCT(1, DATETIME '2023-11-01 09:34:01', 74, 'INACTIVE'),STRUCT(4, DATETIME '2023-11-01 09:38:01', 80, 'ACTIVE')])",
|
||||
write={
|
||||
"bigquery": "SELECT * FROM UNNEST(CAST([STRUCT(1, CAST('2023-11-01 09:34:01' AS DATETIME), 74, 'INACTIVE'), STRUCT(4, CAST('2023-11-01 09:38:01' AS DATETIME), 80, 'ACTIVE')] AS ARRAY<STRUCT<device_id INT64, time DATETIME, signal INT64, state STRING>>))",
|
||||
"duckdb": "SELECT * FROM UNNEST(CAST([ROW(1, CAST('2023-11-01 09:34:01' AS TIMESTAMP), 74, 'INACTIVE'), ROW(4, CAST('2023-11-01 09:38:01' AS TIMESTAMP), 80, 'ACTIVE')] AS STRUCT(device_id BIGINT, time TIMESTAMP, signal BIGINT, state TEXT)[]))",
|
||||
"duckdb": "SELECT * FROM (SELECT UNNEST(CAST([ROW(1, CAST('2023-11-01 09:34:01' AS TIMESTAMP), 74, 'INACTIVE'), ROW(4, CAST('2023-11-01 09:38:01' AS TIMESTAMP), 80, 'ACTIVE')] AS STRUCT(device_id BIGINT, time TIMESTAMP, signal BIGINT, state TEXT)[]), max_depth => 2))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
@ -1811,3 +1886,51 @@ OPTIONS (
|
|||
"duckdb": "SELECT CAST(ROW(1, ROW('c_str')) AS STRUCT(a BIGINT, b STRUCT(c TEXT)))",
|
||||
},
|
||||
)
|
||||
|
||||
def test_convert(self):
|
||||
for value, expected in [
|
||||
(datetime.datetime(2023, 1, 1), "CAST('2023-01-01 00:00:00' AS DATETIME)"),
|
||||
(datetime.datetime(2023, 1, 1, 12, 13, 14), "CAST('2023-01-01 12:13:14' AS DATETIME)"),
|
||||
(
|
||||
datetime.datetime(2023, 1, 1, 12, 13, 14, tzinfo=datetime.timezone.utc),
|
||||
"CAST('2023-01-01 12:13:14+00:00' AS TIMESTAMP)",
|
||||
),
|
||||
(
|
||||
pytz.timezone("America/Los_Angeles").localize(
|
||||
datetime.datetime(2023, 1, 1, 12, 13, 14)
|
||||
),
|
||||
"CAST('2023-01-01 12:13:14-08:00' AS TIMESTAMP)",
|
||||
),
|
||||
]:
|
||||
with self.subTest(value):
|
||||
self.assertEqual(exp.convert(value).sql(dialect=self.dialect), expected)
|
||||
|
||||
def test_unnest(self):
|
||||
self.validate_all(
|
||||
"SELECT name, laps FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps), STRUCT('Makhloufi' AS name, [24.5, 25.4, 26.6, 26.1] AS laps)])",
|
||||
write={
|
||||
"bigquery": "SELECT name, laps FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps), STRUCT('Makhloufi' AS name, [24.5, 25.4, 26.6, 26.1] AS laps)])",
|
||||
"duckdb": "SELECT name, laps FROM (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}, {'name': 'Makhloufi', 'laps': [24.5, 25.4, 26.6, 26.1]}], max_depth => 2))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"WITH Races AS (SELECT '800M' AS race) SELECT race, name, laps FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)])",
|
||||
write={
|
||||
"bigquery": "WITH Races AS (SELECT '800M' AS race) SELECT race, name, laps FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)])",
|
||||
"duckdb": "WITH Races AS (SELECT '800M' AS race) SELECT race, name, laps FROM Races AS r CROSS JOIN (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}], max_depth => 2))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT participant FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
|
||||
write={
|
||||
"bigquery": "SELECT participant FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
|
||||
"duckdb": "SELECT participant FROM (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}], max_depth => 2)) AS participant",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"WITH Races AS (SELECT '800M' AS race) SELECT race, participant FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
|
||||
write={
|
||||
"bigquery": "WITH Races AS (SELECT '800M' AS race) SELECT race, participant FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
|
||||
"duckdb": "WITH Races AS (SELECT '800M' AS race) SELECT race, participant FROM Races AS r CROSS JOIN (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}], max_depth => 2)) AS participant",
|
||||
},
|
||||
)
|
||||
|
|
|
@ -1,4 +1,7 @@
|
|||
from datetime import date
|
||||
from sqlglot import exp, parse_one
|
||||
from sqlglot.dialects import ClickHouse
|
||||
from sqlglot.expressions import convert
|
||||
from tests.dialects.test_dialect import Validator
|
||||
from sqlglot.errors import ErrorLevel
|
||||
|
||||
|
@ -7,27 +10,27 @@ class TestClickhouse(Validator):
|
|||
dialect = "clickhouse"
|
||||
|
||||
def test_clickhouse(self):
|
||||
self.validate_identity("SELECT toFloat(like)")
|
||||
self.validate_identity("SELECT like")
|
||||
for string_type_enum in ClickHouse.Generator.STRING_TYPE_MAPPING:
|
||||
self.validate_identity(f"CAST(x AS {string_type_enum.value})", "CAST(x AS String)")
|
||||
|
||||
string_types = [
|
||||
"BLOB",
|
||||
"LONGBLOB",
|
||||
"LONGTEXT",
|
||||
"MEDIUMBLOB",
|
||||
"MEDIUMTEXT",
|
||||
"TINYBLOB",
|
||||
"TINYTEXT",
|
||||
"VARCHAR(255)",
|
||||
]
|
||||
# Arrays, maps and tuples can't be Nullable in ClickHouse
|
||||
for non_nullable_type in ("ARRAY<INT>", "MAP<INT, INT>", "STRUCT(a: INT)"):
|
||||
try_cast = parse_one(f"TRY_CAST(x AS {non_nullable_type})")
|
||||
target_type = try_cast.to.sql("clickhouse")
|
||||
self.assertEqual(try_cast.sql("clickhouse"), f"CAST(x AS {target_type})")
|
||||
|
||||
for string_type in string_types:
|
||||
self.validate_identity(f"CAST(x AS {string_type})", "CAST(x AS String)")
|
||||
for nullable_type in ("INT", "UINT", "BIGINT", "FLOAT", "DOUBLE", "TEXT", "DATE", "UUID"):
|
||||
try_cast = parse_one(f"TRY_CAST(x AS {nullable_type})")
|
||||
target_type = exp.DataType.build(nullable_type, dialect="clickhouse").sql("clickhouse")
|
||||
self.assertEqual(try_cast.sql("clickhouse"), f"CAST(x AS Nullable({target_type}))")
|
||||
|
||||
expr = parse_one("count(x)")
|
||||
self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)")
|
||||
self.assertIsNone(expr._meta)
|
||||
|
||||
self.validate_identity("@macro").assert_is(exp.Parameter).this.assert_is(exp.Var)
|
||||
self.validate_identity("SELECT toFloat(like)")
|
||||
self.validate_identity("SELECT like")
|
||||
self.validate_identity("SELECT STR_TO_DATE(str, fmt, tz)")
|
||||
self.validate_identity("SELECT STR_TO_DATE('05 12 2000', '%d %m %Y')")
|
||||
self.validate_identity("SELECT EXTRACT(YEAR FROM toDateTime('2023-02-01'))")
|
||||
|
@ -40,7 +43,7 @@ class TestClickhouse(Validator):
|
|||
self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 0.01)")
|
||||
self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 1 / 10 OFFSET 1 / 2)")
|
||||
self.validate_identity("SELECT sum(foo * bar) FROM bla SAMPLE 10000000")
|
||||
self.validate_identity("CAST(x AS Nested(ID UInt32, Serial UInt32, EventTime DATETIME))")
|
||||
self.validate_identity("CAST(x AS Nested(ID UInt32, Serial UInt32, EventTime DateTime))")
|
||||
self.validate_identity("CAST(x AS Enum('hello' = 1, 'world' = 2))")
|
||||
self.validate_identity("CAST(x AS Enum('hello', 'world'))")
|
||||
self.validate_identity("CAST(x AS Enum('hello' = 1, 'world'))")
|
||||
|
@ -79,7 +82,8 @@ class TestClickhouse(Validator):
|
|||
self.validate_identity("SELECT * FROM foo WHERE x GLOBAL IN (SELECT * FROM bar)")
|
||||
self.validate_identity("position(haystack, needle)")
|
||||
self.validate_identity("position(haystack, needle, position)")
|
||||
self.validate_identity("CAST(x AS DATETIME)")
|
||||
self.validate_identity("CAST(x AS DATETIME)", "CAST(x AS DateTime)")
|
||||
self.validate_identity("CAST(x AS TIMESTAMPTZ)", "CAST(x AS DateTime)")
|
||||
self.validate_identity("CAST(x as MEDIUMINT)", "CAST(x AS Int32)")
|
||||
self.validate_identity("SELECT arrayJoin([1, 2, 3] AS src) AS dst, 'Hello', src")
|
||||
self.validate_identity("""SELECT JSONExtractString('{"x": {"y": 1}}', 'x', 'y')""")
|
||||
|
@ -207,11 +211,16 @@ class TestClickhouse(Validator):
|
|||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01' AS TIMESTAMP) + INTERVAL '500' MICROSECOND",
|
||||
"SELECT CAST('2020-01-01' AS Nullable(DateTime)) + INTERVAL '500' MICROSECOND",
|
||||
read={
|
||||
"duckdb": "SELECT TIMESTAMP '2020-01-01' + INTERVAL '500 us'",
|
||||
"postgres": "SELECT TIMESTAMP '2020-01-01' + INTERVAL '500 us'",
|
||||
},
|
||||
write={
|
||||
"clickhouse": "SELECT CAST('2020-01-01' AS Nullable(DateTime)) + INTERVAL '500' MICROSECOND",
|
||||
"duckdb": "SELECT CAST('2020-01-01' AS DATETIME) + INTERVAL '500' MICROSECOND",
|
||||
"postgres": "SELECT CAST('2020-01-01' AS TIMESTAMP) + INTERVAL '500 MICROSECOND'",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CURRENT_DATE()",
|
||||
|
@ -471,6 +480,47 @@ class TestClickhouse(Validator):
|
|||
parse_one("Tuple(select Int64)", into=exp.DataType, read="clickhouse"), exp.DataType
|
||||
)
|
||||
|
||||
self.validate_identity("INSERT INTO t (col1, col2) VALUES ('abcd', 1234)")
|
||||
self.validate_all(
|
||||
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
|
||||
read={
|
||||
# looks like values table function, but should be parsed as VALUES block
|
||||
"clickhouse": "INSERT INTO t (col1, col2) values('abcd', 1234)"
|
||||
},
|
||||
write={
|
||||
"clickhouse": "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
|
||||
"postgres": "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_clickhouse_values(self):
|
||||
values = exp.select("*").from_(
|
||||
exp.values([exp.tuple_(1, 2, 3)], alias="subq", columns=["a", "b", "c"])
|
||||
)
|
||||
self.assertEqual(
|
||||
values.sql("clickhouse"),
|
||||
"SELECT * FROM (SELECT 1 AS a, 2 AS b, 3 AS c) AS subq",
|
||||
)
|
||||
|
||||
self.validate_identity("INSERT INTO t (col1, col2) VALUES ('abcd', 1234)")
|
||||
self.validate_identity(
|
||||
"INSERT INTO t (col1, col2) FORMAT Values('abcd', 1234)",
|
||||
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT col FROM (SELECT 1 AS col) AS _t",
|
||||
read={
|
||||
"duckdb": "SELECT col FROM (VALUES (1)) AS _t(col)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT col1, col2 FROM (SELECT 1 AS col1, 2 AS col2 UNION ALL SELECT 3, 4) AS _t",
|
||||
read={
|
||||
"duckdb": "SELECT col1, col2 FROM (VALUES (1, 2), (3, 4)) AS _t(col1, col2)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_cte(self):
|
||||
self.validate_identity("WITH 'x' AS foo SELECT foo")
|
||||
self.validate_identity("WITH ['c'] AS field_names SELECT field_names")
|
||||
|
@ -531,7 +581,7 @@ class TestClickhouse(Validator):
|
|||
self.validate_all(
|
||||
"SELECT {abc: UInt32}, {b: String}, {c: DateTime},{d: Map(String, Array(UInt8))}, {e: Tuple(UInt8, String)}",
|
||||
write={
|
||||
"clickhouse": "SELECT {abc: UInt32}, {b: String}, {c: DATETIME}, {d: Map(String, Array(UInt8))}, {e: Tuple(UInt8, String)}",
|
||||
"clickhouse": "SELECT {abc: UInt32}, {b: String}, {c: DateTime}, {d: Map(String, Array(UInt8))}, {e: Tuple(UInt8, String)}",
|
||||
"": "SELECT :abc, :b, :c, :d, :e",
|
||||
},
|
||||
)
|
||||
|
@ -562,14 +612,77 @@ class TestClickhouse(Validator):
|
|||
)
|
||||
|
||||
def test_ddl(self):
|
||||
db_table_expr = exp.Table(this=None, db=exp.to_identifier("foo"), catalog=None)
|
||||
create_with_cluster = exp.Create(
|
||||
this=db_table_expr,
|
||||
kind="DATABASE",
|
||||
properties=exp.Properties(expressions=[exp.OnCluster(this=exp.to_identifier("c"))]),
|
||||
)
|
||||
self.assertEqual(create_with_cluster.sql("clickhouse"), "CREATE DATABASE foo ON CLUSTER c")
|
||||
|
||||
ctas_with_comment = exp.Create(
|
||||
this=exp.table_("foo"),
|
||||
kind="TABLE",
|
||||
expression=exp.select("*").from_("db.other_table"),
|
||||
properties=exp.Properties(
|
||||
expressions=[
|
||||
exp.EngineProperty(this=exp.var("Memory")),
|
||||
exp.SchemaCommentProperty(this=exp.Literal.string("foo")),
|
||||
],
|
||||
),
|
||||
)
|
||||
self.assertEqual(
|
||||
ctas_with_comment.sql("clickhouse"),
|
||||
"CREATE TABLE foo ENGINE=Memory AS (SELECT * FROM db.other_table) COMMENT 'foo'",
|
||||
)
|
||||
|
||||
self.validate_identity("""CREATE TABLE ip_data (ip4 IPv4, ip6 IPv6) ENGINE=TinyLog()""")
|
||||
self.validate_identity("""CREATE TABLE dates (dt1 Date32) ENGINE=TinyLog()""")
|
||||
self.validate_identity("CREATE TABLE named_tuples (a Tuple(select String, i Int64))")
|
||||
self.validate_identity("""CREATE TABLE t (a String) EMPTY AS SELECT * FROM dummy""")
|
||||
self.validate_identity(
|
||||
"CREATE TABLE t1 (a String EPHEMERAL, b String EPHEMERAL func(), c String MATERIALIZED func(), d String ALIAS func()) ENGINE=TinyLog()"
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE TABLE t (a String, b String, c UInt64, PROJECTION p1 (SELECT a, sum(c) GROUP BY a, b), PROJECTION p2 (SELECT b, sum(c) GROUP BY b)) ENGINE=MergeTree()"
|
||||
)
|
||||
self.validate_identity(
|
||||
"""CREATE TABLE xyz (ts DateTime, data String) ENGINE=MergeTree() ORDER BY ts SETTINGS index_granularity = 8192 COMMENT '{"key": "value"}'"""
|
||||
)
|
||||
self.validate_identity(
|
||||
"INSERT INTO FUNCTION s3('a', 'b', 'c', 'd', 'e') PARTITION BY CONCAT(s1, s2, s3, s4) SETTINGS set1 = 1, set2 = '2' SELECT * FROM some_table SETTINGS foo = 3"
|
||||
)
|
||||
self.validate_identity(
|
||||
'CREATE TABLE data5 ("x" UInt32, "y" UInt32) ENGINE=MergeTree ORDER BY (round(y / 1000000000), cityHash64(x)) SAMPLE BY cityHash64(x)'
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE TABLE foo (x UInt32) TTL time_column + INTERVAL '1' MONTH DELETE WHERE column = 'value'"
|
||||
)
|
||||
self.validate_identity("CREATE TABLE named_tuples (a Tuple(select String, i Int64))")
|
||||
self.validate_identity(
|
||||
"CREATE TABLE a ENGINE=Memory AS SELECT 1 AS c COMMENT 'foo'",
|
||||
"CREATE TABLE a ENGINE=Memory AS (SELECT 1 AS c) COMMENT 'foo'",
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CREATE DATABASE x",
|
||||
read={
|
||||
"duckdb": "CREATE SCHEMA x",
|
||||
},
|
||||
write={
|
||||
"clickhouse": "CREATE DATABASE x",
|
||||
"duckdb": "CREATE SCHEMA x",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"DROP DATABASE x",
|
||||
read={
|
||||
"duckdb": "DROP SCHEMA x",
|
||||
},
|
||||
write={
|
||||
"clickhouse": "DROP DATABASE x",
|
||||
"duckdb": "DROP SCHEMA x",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"""
|
||||
CREATE TABLE example1 (
|
||||
|
@ -583,7 +696,7 @@ class TestClickhouse(Validator):
|
|||
""",
|
||||
write={
|
||||
"clickhouse": """CREATE TABLE example1 (
|
||||
timestamp DATETIME,
|
||||
timestamp DateTime,
|
||||
x UInt32 TTL now() + INTERVAL '1' MONTH,
|
||||
y String TTL timestamp + INTERVAL '1' DAY,
|
||||
z String
|
||||
|
@ -661,7 +774,7 @@ SETTINGS
|
|||
""",
|
||||
write={
|
||||
"clickhouse": """CREATE TABLE example_table (
|
||||
d DATETIME,
|
||||
d DateTime,
|
||||
a Int32
|
||||
)
|
||||
ENGINE=MergeTree
|
||||
|
@ -688,7 +801,7 @@ TTL
|
|||
""",
|
||||
write={
|
||||
"clickhouse": """CREATE TABLE table_with_where (
|
||||
d DATETIME,
|
||||
d DateTime,
|
||||
a Int32
|
||||
)
|
||||
ENGINE=MergeTree
|
||||
|
@ -716,7 +829,7 @@ WHERE
|
|||
""",
|
||||
write={
|
||||
"clickhouse": """CREATE TABLE table_for_recompression (
|
||||
d DATETIME,
|
||||
d DateTime,
|
||||
key UInt64,
|
||||
value String
|
||||
)
|
||||
|
@ -748,7 +861,7 @@ SETTINGS
|
|||
""",
|
||||
write={
|
||||
"clickhouse": """CREATE TABLE table_for_aggregation (
|
||||
d DATETIME,
|
||||
d DateTime,
|
||||
k1 Int32,
|
||||
k2 Int32,
|
||||
x Int32,
|
||||
|
@ -855,8 +968,6 @@ LIFETIME(MIN 0 MAX 0)""",
|
|||
},
|
||||
pretty=True,
|
||||
)
|
||||
self.validate_identity("""CREATE TABLE ip_data (ip4 IPv4, ip6 IPv6) ENGINE=TinyLog()""")
|
||||
self.validate_identity("""CREATE TABLE dates (dt1 Date32) ENGINE=TinyLog()""")
|
||||
self.validate_all(
|
||||
"""
|
||||
CREATE TABLE t (
|
||||
|
@ -873,12 +984,6 @@ LIFETIME(MIN 0 MAX 0)""",
|
|||
},
|
||||
pretty=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE TABLE t1 (a String EPHEMERAL, b String EPHEMERAL func(), c String MATERIALIZED func(), d String ALIAS func()) ENGINE=TinyLog()"
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE TABLE t (a String, b String, c UInt64, PROJECTION p1 (SELECT a, sum(c) GROUP BY a, b), PROJECTION p2 (SELECT b, sum(c) GROUP BY b)) ENGINE=MergeTree()"
|
||||
)
|
||||
|
||||
def test_agg_functions(self):
|
||||
def extract_agg_func(query):
|
||||
|
@ -934,3 +1039,8 @@ LIFETIME(MIN 0 MAX 0)""",
|
|||
f"SELECT {func_alias}(SECOND, 1, bar)",
|
||||
f"SELECT {func_name}(SECOND, 1, bar)",
|
||||
)
|
||||
|
||||
def test_convert(self):
|
||||
self.assertEqual(
|
||||
convert(date(2020, 1, 1)).sql(dialect=self.dialect), "toDate('2020-01-01')"
|
||||
)
|
||||
|
|
|
@ -256,3 +256,11 @@ class TestDatabricks(Validator):
|
|||
"databricks": "WITH x AS (SELECT 1) SELECT * FROM x",
|
||||
},
|
||||
)
|
||||
|
||||
def test_streaming_tables(self):
|
||||
self.validate_identity(
|
||||
"CREATE STREAMING TABLE raw_data AS SELECT * FROM STREAM READ_FILES('abfss://container@storageAccount.dfs.core.windows.net/base/path')"
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE OR REFRESH STREAMING TABLE csv_data (id INT, ts TIMESTAMP, event STRING) AS SELECT * FROM STREAM READ_FILES('s3://bucket/path', format => 'csv', schema => 'id int, ts timestamp, event string')"
|
||||
)
|
||||
|
|
|
@ -160,7 +160,7 @@ class TestDialect(Validator):
|
|||
"CAST(a AS TEXT)",
|
||||
write={
|
||||
"bigquery": "CAST(a AS STRING)",
|
||||
"clickhouse": "CAST(a AS String)",
|
||||
"clickhouse": "CAST(a AS Nullable(String))",
|
||||
"drill": "CAST(a AS VARCHAR)",
|
||||
"duckdb": "CAST(a AS TEXT)",
|
||||
"materialize": "CAST(a AS TEXT)",
|
||||
|
@ -181,7 +181,7 @@ class TestDialect(Validator):
|
|||
"CAST(a AS BINARY(4))",
|
||||
write={
|
||||
"bigquery": "CAST(a AS BYTES)",
|
||||
"clickhouse": "CAST(a AS BINARY(4))",
|
||||
"clickhouse": "CAST(a AS Nullable(BINARY(4)))",
|
||||
"drill": "CAST(a AS VARBINARY(4))",
|
||||
"duckdb": "CAST(a AS BLOB(4))",
|
||||
"materialize": "CAST(a AS BYTEA(4))",
|
||||
|
@ -201,7 +201,7 @@ class TestDialect(Validator):
|
|||
"CAST(a AS VARBINARY(4))",
|
||||
write={
|
||||
"bigquery": "CAST(a AS BYTES)",
|
||||
"clickhouse": "CAST(a AS String)",
|
||||
"clickhouse": "CAST(a AS Nullable(String))",
|
||||
"duckdb": "CAST(a AS BLOB(4))",
|
||||
"materialize": "CAST(a AS BYTEA(4))",
|
||||
"mysql": "CAST(a AS VARBINARY(4))",
|
||||
|
@ -219,19 +219,19 @@ class TestDialect(Validator):
|
|||
self.validate_all(
|
||||
"CAST(MAP('a', '1') AS MAP(TEXT, TEXT))",
|
||||
write={
|
||||
"clickhouse": "CAST(map('a', '1') AS Map(String, String))",
|
||||
"clickhouse": "CAST(map('a', '1') AS Map(String, Nullable(String)))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CAST(ARRAY(1, 2) AS ARRAY<TINYINT>)",
|
||||
write={
|
||||
"clickhouse": "CAST([1, 2] AS Array(Int8))",
|
||||
"clickhouse": "CAST([1, 2] AS Array(Nullable(Int8)))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CAST((1, 2) AS STRUCT<a: TINYINT, b: SMALLINT, c: INT, d: BIGINT>)",
|
||||
"CAST((1, 2, 3, 4) AS STRUCT<a: TINYINT, b: SMALLINT, c: INT, d: BIGINT>)",
|
||||
write={
|
||||
"clickhouse": "CAST((1, 2) AS Tuple(a Int8, b Int16, c Int32, d Int64))",
|
||||
"clickhouse": "CAST((1, 2, 3, 4) AS Tuple(a Nullable(Int8), b Nullable(Int16), c Nullable(Int32), d Nullable(Int64)))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
@ -327,20 +327,10 @@ class TestDialect(Validator):
|
|||
"postgres": "CAST(a AS DOUBLE PRECISION)",
|
||||
"redshift": "CAST(a AS DOUBLE PRECISION)",
|
||||
},
|
||||
write={
|
||||
"duckdb": "CAST(a AS DOUBLE)",
|
||||
"drill": "CAST(a AS DOUBLE)",
|
||||
"postgres": "CAST(a AS DOUBLE PRECISION)",
|
||||
"redshift": "CAST(a AS DOUBLE PRECISION)",
|
||||
"doris": "CAST(a AS DOUBLE)",
|
||||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(a AS DOUBLE)",
|
||||
write={
|
||||
"bigquery": "CAST(a AS FLOAT64)",
|
||||
"clickhouse": "CAST(a AS Float64)",
|
||||
"clickhouse": "CAST(a AS Nullable(Float64))",
|
||||
"doris": "CAST(a AS DOUBLE)",
|
||||
"drill": "CAST(a AS DOUBLE)",
|
||||
"duckdb": "CAST(a AS DOUBLE)",
|
||||
"materialize": "CAST(a AS DOUBLE PRECISION)",
|
||||
|
@ -592,7 +582,7 @@ class TestDialect(Validator):
|
|||
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'yyyy-MM-ddTHH:mm:ss')) AS TIMESTAMP)",
|
||||
"presto": "DATE_PARSE(x, '%Y-%m-%dT%T')",
|
||||
"drill": "TO_TIMESTAMP(x, 'yyyy-MM-dd''T''HH:mm:ss')",
|
||||
"redshift": "TO_TIMESTAMP(x, 'YYYY-MM-DDTHH:MI:SS')",
|
||||
"redshift": "TO_TIMESTAMP(x, 'YYYY-MM-DDTHH24:MI:SS')",
|
||||
"spark": "TO_TIMESTAMP(x, 'yyyy-MM-ddTHH:mm:ss')",
|
||||
},
|
||||
)
|
||||
|
@ -647,14 +637,66 @@ class TestDialect(Validator):
|
|||
self.validate_all(
|
||||
"TIME_STR_TO_TIME('2020-01-01')",
|
||||
write={
|
||||
"drill": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"bigquery": "CAST('2020-01-01' AS DATETIME)",
|
||||
"databricks": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"duckdb": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"tsql": "CAST('2020-01-01' AS DATETIME2)",
|
||||
"mysql": "CAST('2020-01-01' AS DATETIME)",
|
||||
"postgres": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"redshift": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"snowflake": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"spark": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"trino": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"clickhouse": "CAST('2020-01-01' AS Nullable(DateTime))",
|
||||
"drill": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"hive": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"presto": "CAST('2020-01-01' AS TIMESTAMP)",
|
||||
"sqlite": "'2020-01-01'",
|
||||
"doris": "CAST('2020-01-01' AS DATETIME)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TIME_STR_TO_TIME('2020-01-01 12:13:14-08:00', 'America/Los_Angeles')",
|
||||
write={
|
||||
"bigquery": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
|
||||
"databricks": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
|
||||
"duckdb": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMPTZ)",
|
||||
"tsql": "CAST('2020-01-01 12:13:14-08:00' AS DATETIMEOFFSET) AT TIME ZONE 'UTC'",
|
||||
"mysql": "CAST('2020-01-01 12:13:14-08:00' AS DATETIME)",
|
||||
"postgres": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMPTZ)",
|
||||
"redshift": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP WITH TIME ZONE)",
|
||||
"snowflake": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMPTZ)",
|
||||
"spark": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
|
||||
"trino": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP WITH TIME ZONE)",
|
||||
"clickhouse": "CAST('2020-01-01 12:13:14' AS Nullable(DateTime('America/Los_Angeles')))",
|
||||
"drill": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
|
||||
"hive": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
|
||||
"presto": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP WITH TIME ZONE)",
|
||||
"sqlite": "'2020-01-01 12:13:14-08:00'",
|
||||
"doris": "CAST('2020-01-01 12:13:14-08:00' AS DATETIME)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TIME_STR_TO_TIME(col, 'America/Los_Angeles')",
|
||||
write={
|
||||
"bigquery": "CAST(col AS TIMESTAMP)",
|
||||
"databricks": "CAST(col AS TIMESTAMP)",
|
||||
"duckdb": "CAST(col AS TIMESTAMPTZ)",
|
||||
"tsql": "CAST(col AS DATETIMEOFFSET) AT TIME ZONE 'UTC'",
|
||||
"mysql": "CAST(col AS DATETIME)",
|
||||
"postgres": "CAST(col AS TIMESTAMPTZ)",
|
||||
"redshift": "CAST(col AS TIMESTAMP WITH TIME ZONE)",
|
||||
"snowflake": "CAST(col AS TIMESTAMPTZ)",
|
||||
"spark": "CAST(col AS TIMESTAMP)",
|
||||
"trino": "CAST(col AS TIMESTAMP WITH TIME ZONE)",
|
||||
"clickhouse": "CAST(col AS Nullable(DateTime('America/Los_Angeles')))",
|
||||
"drill": "CAST(col AS TIMESTAMP)",
|
||||
"hive": "CAST(col AS TIMESTAMP)",
|
||||
"presto": "CAST(col AS TIMESTAMP WITH TIME ZONE)",
|
||||
"sqlite": "col",
|
||||
"doris": "CAST(col AS DATETIME)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TIME_STR_TO_UNIX('2020-01-01')",
|
||||
write={
|
||||
|
@ -2115,6 +2157,17 @@ SELECT
|
|||
},
|
||||
)
|
||||
|
||||
# needs to preserve the target alias in then WHEN condition but not in the THEN clause
|
||||
self.validate_all(
|
||||
"""MERGE INTO foo AS target USING (SELECT a, b FROM tbl) AS src ON src.a = target.a
|
||||
WHEN MATCHED AND target.a <> src.a THEN UPDATE SET target.b = 'FOO'
|
||||
WHEN NOT MATCHED THEN INSERT (target.a, target.b) VALUES (src.a, src.b)""",
|
||||
write={
|
||||
"trino": """MERGE INTO foo AS target USING (SELECT a, b FROM tbl) AS src ON src.a = target.a WHEN MATCHED AND target.a <> src.a THEN UPDATE SET b = 'FOO' WHEN NOT MATCHED THEN INSERT (a, b) VALUES (src.a, src.b)""",
|
||||
"postgres": """MERGE INTO foo AS target USING (SELECT a, b FROM tbl) AS src ON src.a = target.a WHEN MATCHED AND target.a <> src.a THEN UPDATE SET b = 'FOO' WHEN NOT MATCHED THEN INSERT (a, b) VALUES (src.a, src.b)""",
|
||||
},
|
||||
)
|
||||
|
||||
def test_substring(self):
|
||||
self.validate_all(
|
||||
"SUBSTR('123456', 2, 3)",
|
||||
|
@ -2603,3 +2656,46 @@ FROM subquery2""",
|
|||
"trino": f"SELECT {pad_func}('bar', 5, ' ')",
|
||||
},
|
||||
)
|
||||
|
||||
def test_generate_date_array(self):
|
||||
self.validate_all(
|
||||
"SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK))",
|
||||
write={
|
||||
"bigquery": "SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
|
||||
"databricks": "SELECT * FROM EXPLODE(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
|
||||
"duckdb": "SELECT * FROM UNNEST(CAST(GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (7 * INTERVAL '1' DAY)) AS DATE[]))",
|
||||
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates",
|
||||
"postgres": "SELECT * FROM (SELECT CAST(value AS DATE) FROM GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1 WEEK') AS value) AS _unnested_generate_series",
|
||||
"presto": "SELECT * FROM UNNEST(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (1 * INTERVAL '7' DAY)))",
|
||||
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates",
|
||||
"snowflake": "SELECT * FROM (SELECT DATEADD(WEEK, CAST(value AS INT), CAST('2020-01-01' AS DATE)) AS value FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (DATEDIFF(WEEK, CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE)) + 1 - 1) + 1))) AS _u(seq, key, path, index, value, this))",
|
||||
"spark": "SELECT * FROM EXPLODE(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
|
||||
"trino": "SELECT * FROM UNNEST(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (1 * INTERVAL '7' DAY)))",
|
||||
"tsql": "WITH _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates) AS _generated_dates",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"WITH dates AS (SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK))) SELECT * FROM dates",
|
||||
write={
|
||||
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)), dates AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates) SELECT * FROM dates",
|
||||
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), dates AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates) SELECT * FROM dates",
|
||||
"tsql": "WITH _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), dates AS (SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates) AS _generated_dates) SELECT * FROM dates",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"WITH dates1 AS (SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK))), dates2 AS (SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-03-01', INTERVAL 1 MONTH))) SELECT * FROM dates1 CROSS JOIN dates2",
|
||||
write={
|
||||
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)), _generated_dates_1(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 MONTH) AS DATE) FROM _generated_dates_1 WHERE CAST(DATE_ADD(date_value, INTERVAL 1 MONTH) AS DATE) <= CAST('2020-03-01' AS DATE)), dates1 AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates), dates2 AS (SELECT * FROM (SELECT date_value FROM _generated_dates_1) AS _generated_dates_1) SELECT * FROM dates1 CROSS JOIN dates2",
|
||||
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), _generated_dates_1(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(MONTH, 1, date_value) AS DATE) FROM _generated_dates_1 WHERE CAST(DATEADD(MONTH, 1, date_value) AS DATE) <= CAST('2020-03-01' AS DATE)), dates1 AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates), dates2 AS (SELECT * FROM (SELECT date_value FROM _generated_dates_1) AS _generated_dates_1) SELECT * FROM dates1 CROSS JOIN dates2",
|
||||
"tsql": "WITH _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), _generated_dates_1(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(MONTH, 1, date_value) AS DATE) FROM _generated_dates_1 WHERE CAST(DATEADD(MONTH, 1, date_value) AS DATE) <= CAST('2020-03-01' AS DATE)), dates1 AS (SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates) AS _generated_dates), dates2 AS (SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates_1) AS _generated_dates_1) SELECT * FROM dates1 CROSS JOIN dates2",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK)) AS _q(date_week)",
|
||||
write={
|
||||
"mysql": "WITH RECURSIVE _generated_dates(date_week) AS (SELECT CAST('2020-01-01' AS DATE) AS date_week UNION ALL SELECT CAST(DATE_ADD(date_week, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_week, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_week FROM _generated_dates) AS _generated_dates",
|
||||
"redshift": "WITH RECURSIVE _generated_dates(date_week) AS (SELECT CAST('2020-01-01' AS DATE) AS date_week UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_week) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_week) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_week FROM _generated_dates) AS _generated_dates",
|
||||
"snowflake": "SELECT * FROM (SELECT DATEADD(WEEK, CAST(date_week AS INT), CAST('2020-01-01' AS DATE)) AS date_week FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (DATEDIFF(WEEK, CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE)) + 1 - 1) + 1))) AS _q(seq, key, path, index, date_week, this)) AS _q(date_week)",
|
||||
"tsql": "WITH _generated_dates(date_week) AS (SELECT CAST('2020-01-01' AS DATE) AS date_week UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_week) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_week) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_week AS date_week FROM _generated_dates) AS _generated_dates",
|
||||
},
|
||||
)
|
||||
|
|
|
@ -8,8 +8,6 @@ class TestDuckDB(Validator):
|
|||
dialect = "duckdb"
|
||||
|
||||
def test_duckdb(self):
|
||||
self.validate_identity("x::int[3]", "CAST(x AS INT[3])")
|
||||
|
||||
with self.assertRaises(ParseError):
|
||||
parse_one("1 //", read="duckdb")
|
||||
|
||||
|
@ -34,31 +32,6 @@ class TestDuckDB(Validator):
|
|||
"mysql": "SELECT `straight_join`",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
|
||||
read={
|
||||
"duckdb": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
|
||||
"snowflake": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMPNTZ)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
|
||||
read={
|
||||
"duckdb": "SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
|
||||
"mysql": "SELECT DATE '2020-01-01' + INTERVAL day_offset DAY FROM t",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('09:05:03' AS TIME) + INTERVAL 2 HOUR",
|
||||
read={
|
||||
"bigquery": "SELECT TIME_ADD(CAST('09:05:03' AS TIME), INTERVAL 2 HOUR)",
|
||||
"snowflake": "SELECT TIMEADD(HOUR, 2, TO_TIME('09:05:03'))",
|
||||
},
|
||||
write={
|
||||
"duckdb": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2' HOUR",
|
||||
"snowflake": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2 HOUR'",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
'STRUCT_PACK("a b" := 1)',
|
||||
write={
|
||||
|
@ -112,7 +85,9 @@ class TestDuckDB(Validator):
|
|||
|
||||
self.validate_all(
|
||||
"CREATE TEMPORARY FUNCTION f1(a, b) AS (a + b)",
|
||||
read={"bigquery": "CREATE TEMP FUNCTION f1(a INT64, b INT64) AS (a + b)"},
|
||||
read={
|
||||
"bigquery": "CREATE TEMP FUNCTION f1(a INT64, b INT64) AS (a + b)",
|
||||
},
|
||||
)
|
||||
self.validate_identity("SELECT 1 WHERE x > $1")
|
||||
self.validate_identity("SELECT 1 WHERE x > $name")
|
||||
|
@ -128,13 +103,17 @@ class TestDuckDB(Validator):
|
|||
)
|
||||
|
||||
self.validate_all(
|
||||
"{'a': 1, 'b': '2'}", write={"presto": "CAST(ROW(1, '2') AS ROW(a INTEGER, b VARCHAR))"}
|
||||
"{'a': 1, 'b': '2'}",
|
||||
write={
|
||||
"presto": "CAST(ROW(1, '2') AS ROW(a INTEGER, b VARCHAR))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"struct_pack(a := 1, b := 2)",
|
||||
write={"presto": "CAST(ROW(1, 2) AS ROW(a INTEGER, b INTEGER))"},
|
||||
write={
|
||||
"presto": "CAST(ROW(1, 2) AS ROW(a INTEGER, b INTEGER))",
|
||||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"struct_pack(a := 1, b := x)",
|
||||
write={
|
||||
|
@ -818,6 +797,9 @@ class TestDuckDB(Validator):
|
|||
)
|
||||
|
||||
self.validate_identity("SELECT LENGTH(foo)")
|
||||
self.validate_identity("SELECT ARRAY[1, 2, 3]", "SELECT [1, 2, 3]")
|
||||
|
||||
self.validate_identity("SELECT * FROM (DESCRIBE t)")
|
||||
|
||||
def test_array_index(self):
|
||||
with self.assertLogs(helper_logger) as cm:
|
||||
|
@ -909,12 +891,12 @@ class TestDuckDB(Validator):
|
|||
"EPOCH_MS(x)",
|
||||
write={
|
||||
"bigquery": "TIMESTAMP_MILLIS(x)",
|
||||
"clickhouse": "fromUnixTimestamp64Milli(CAST(x AS Nullable(Int64)))",
|
||||
"duckdb": "EPOCH_MS(x)",
|
||||
"mysql": "FROM_UNIXTIME(x / POWER(10, 3))",
|
||||
"postgres": "TO_TIMESTAMP(CAST(x AS DOUBLE PRECISION) / 10 ^ 3)",
|
||||
"presto": "FROM_UNIXTIME(CAST(x AS DOUBLE) / POW(10, 3))",
|
||||
"spark": "TIMESTAMP_MILLIS(x)",
|
||||
"clickhouse": "fromUnixTimestamp64Milli(CAST(x AS Int64))",
|
||||
"postgres": "TO_TIMESTAMP(CAST(x AS DOUBLE PRECISION) / 10 ^ 3)",
|
||||
"mysql": "FROM_UNIXTIME(x / POWER(10, 3))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
@ -958,7 +940,7 @@ class TestDuckDB(Validator):
|
|||
self.validate_all(
|
||||
"STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
|
||||
write={
|
||||
"bigquery": "PARSE_TIMESTAMP('%-m/%-d/%y %-I:%M %p', x)",
|
||||
"bigquery": "PARSE_TIMESTAMP('%-m/%e/%y %-I:%M %p', x)",
|
||||
"duckdb": "STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
|
||||
"presto": "DATE_PARSE(x, '%c/%e/%y %l:%i %p')",
|
||||
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'M/d/yy h:mm a')) AS TIMESTAMP)",
|
||||
|
@ -1023,6 +1005,7 @@ class TestDuckDB(Validator):
|
|||
self.validate_identity("ARRAY((SELECT id FROM t))")
|
||||
|
||||
def test_cast(self):
|
||||
self.validate_identity("x::int[3]", "CAST(x AS INT[3])")
|
||||
self.validate_identity("CAST(x AS REAL)")
|
||||
self.validate_identity("CAST(x AS UINTEGER)")
|
||||
self.validate_identity("CAST(x AS UBIGINT)")
|
||||
|
@ -1076,6 +1059,39 @@ class TestDuckDB(Validator):
|
|||
"STRUCT_PACK(a := 'b')::STRUCT(a TEXT)",
|
||||
"CAST(ROW('b') AS STRUCT(a TEXT))",
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x AS TIME)",
|
||||
read={
|
||||
"duckdb": "CAST(x AS TIME)",
|
||||
"presto": "CAST(x AS TIME(6))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
|
||||
read={
|
||||
"duckdb": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
|
||||
"snowflake": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMPNTZ)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
|
||||
read={
|
||||
"duckdb": "SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
|
||||
"mysql": "SELECT DATE '2020-01-01' + INTERVAL day_offset DAY FROM t",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('09:05:03' AS TIME) + INTERVAL 2 HOUR",
|
||||
read={
|
||||
"bigquery": "SELECT TIME_ADD(CAST('09:05:03' AS TIME), INTERVAL 2 HOUR)",
|
||||
"snowflake": "SELECT TIMEADD(HOUR, 2, TO_TIME('09:05:03'))",
|
||||
},
|
||||
write={
|
||||
"duckdb": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2' HOUR",
|
||||
"snowflake": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2 HOUR'",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CAST(x AS VARCHAR(5))",
|
||||
write={
|
||||
|
@ -1156,6 +1172,12 @@ class TestDuckDB(Validator):
|
|||
},
|
||||
)
|
||||
|
||||
self.validate_identity("SELECT x::INT[3][3]", "SELECT CAST(x AS INT[3][3])")
|
||||
self.validate_identity(
|
||||
"""SELECT ARRAY[[[1]]]::INT[1][1][1]""",
|
||||
"""SELECT CAST([[[1]]] AS INT[1][1][1])""",
|
||||
)
|
||||
|
||||
def test_encode_decode(self):
|
||||
self.validate_all(
|
||||
"ENCODE(x)",
|
||||
|
|
|
@ -171,6 +171,16 @@ class TestHive(Validator):
|
|||
self.validate_identity(
|
||||
"""CREATE EXTERNAL TABLE `my_table` (`a7` ARRAY<DATE>) ROW FORMAT SERDE 'a' STORED AS INPUTFORMAT 'b' OUTPUTFORMAT 'c' LOCATION 'd' TBLPROPERTIES ('e'='f')"""
|
||||
)
|
||||
self.validate_identity("ALTER VIEW v1 AS SELECT x, UPPER(s) AS s FROM t2")
|
||||
self.validate_identity("ALTER VIEW v1 (c1, c2) AS SELECT x, UPPER(s) AS s FROM t2")
|
||||
self.validate_identity(
|
||||
"ALTER VIEW v7 (c1 COMMENT 'Comment for c1', c2) AS SELECT t1.c1, t1.c2 FROM t1"
|
||||
)
|
||||
self.validate_identity("ALTER VIEW db1.v1 RENAME TO db2.v2")
|
||||
self.validate_identity("ALTER VIEW v1 SET TBLPROPERTIES ('tblp1'='1', 'tblp2'='2')")
|
||||
self.validate_identity(
|
||||
"ALTER VIEW v1 UNSET TBLPROPERTIES ('tblp1', 'tblp2')", check_command_warning=True
|
||||
)
|
||||
|
||||
def test_lateral_view(self):
|
||||
self.validate_all(
|
||||
|
|
|
@ -24,6 +24,18 @@ class TestMySQL(Validator):
|
|||
self.validate_identity("ALTER TABLE t ADD INDEX `i` (`c`)")
|
||||
self.validate_identity("ALTER TABLE t ADD UNIQUE `i` (`c`)")
|
||||
self.validate_identity("ALTER TABLE test_table MODIFY COLUMN test_column LONGTEXT")
|
||||
self.validate_identity("ALTER VIEW v AS SELECT a, b, c, d FROM foo")
|
||||
self.validate_identity("ALTER VIEW v AS SELECT * FROM foo WHERE c > 100")
|
||||
self.validate_identity(
|
||||
"ALTER ALGORITHM = MERGE VIEW v AS SELECT * FROM foo", check_command_warning=True
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER DEFINER = 'admin'@'localhost' VIEW v AS SELECT * FROM foo",
|
||||
check_command_warning=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER SQL SECURITY = DEFINER VIEW v AS SELECT * FROM foo", check_command_warning=True
|
||||
)
|
||||
self.validate_identity(
|
||||
"INSERT INTO things (a, b) VALUES (1, 2) AS new_data ON DUPLICATE KEY UPDATE id = LAST_INSERT_ID(id), a = new_data.a, b = new_data.b"
|
||||
)
|
||||
|
|
|
@ -6,6 +6,7 @@ class TestOracle(Validator):
|
|||
dialect = "oracle"
|
||||
|
||||
def test_oracle(self):
|
||||
self.validate_identity("1 /* /* */")
|
||||
self.validate_all(
|
||||
"SELECT CONNECT_BY_ROOT x y",
|
||||
write={
|
||||
|
@ -13,8 +14,9 @@ class TestOracle(Validator):
|
|||
"oracle": "SELECT CONNECT_BY_ROOT x AS y",
|
||||
},
|
||||
)
|
||||
self.parse_one("ALTER TABLE tbl_name DROP FOREIGN KEY fk_symbol").assert_is(exp.AlterTable)
|
||||
self.parse_one("ALTER TABLE tbl_name DROP FOREIGN KEY fk_symbol").assert_is(exp.Alter)
|
||||
|
||||
self.validate_identity("SYSDATE")
|
||||
self.validate_identity("CREATE GLOBAL TEMPORARY TABLE t AS SELECT * FROM orders")
|
||||
self.validate_identity("CREATE PRIVATE TEMPORARY TABLE t AS SELECT * FROM orders")
|
||||
self.validate_identity("REGEXP_REPLACE('source', 'search')")
|
||||
|
@ -43,6 +45,9 @@ class TestOracle(Validator):
|
|||
self.validate_identity("SELECT COUNT(*) * 10 FROM orders SAMPLE (10) SEED (1)")
|
||||
self.validate_identity("SELECT * FROM V$SESSION")
|
||||
self.validate_identity("SELECT TO_DATE('January 15, 1989, 11:00 A.M.')")
|
||||
self.validate_identity(
|
||||
"SELECT * FROM test UNPIVOT INCLUDE NULLS (value FOR Description IN (col AS 'PREFIX ' || CHR(38) || ' SUFFIX'))"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT last_name, employee_id, manager_id, LEVEL FROM employees START WITH employee_id = 100 CONNECT BY PRIOR employee_id = manager_id ORDER SIBLINGS BY last_name"
|
||||
)
|
||||
|
@ -72,10 +77,6 @@ class TestOracle(Validator):
|
|||
"SELECT JSON_OBJECTAGG(KEY department_name VALUE department_id) FROM dep WHERE id <= 30",
|
||||
"SELECT JSON_OBJECTAGG(department_name: department_id) FROM dep WHERE id <= 30",
|
||||
)
|
||||
self.validate_identity(
|
||||
"SYSDATE",
|
||||
"CURRENT_TIMESTAMP",
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT last_name, department_id, salary, MIN(salary) KEEP (DENSE_RANK FIRST ORDER BY commission_pct) "
|
||||
'OVER (PARTITION BY department_id) AS "Worst", MAX(salary) KEEP (DENSE_RANK LAST ORDER BY commission_pct) '
|
||||
|
@ -88,7 +89,6 @@ class TestOracle(Validator):
|
|||
self.validate_identity(
|
||||
"SELECT * FROM T ORDER BY I OFFSET NVL(:variable1, 10) ROWS FETCH NEXT NVL(:variable2, 10) ROWS ONLY",
|
||||
)
|
||||
self.validate_identity("NVL(x, y)").assert_is(exp.Anonymous)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM t SAMPLE (.25)",
|
||||
"SELECT * FROM t SAMPLE (0.25)",
|
||||
|
@ -98,6 +98,16 @@ class TestOracle(Validator):
|
|||
"SELECT * FROM t START WITH col CONNECT BY NOCYCLE PRIOR col1 = col2"
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT * FROM test WHERE MOD(col1, 4) = 3",
|
||||
read={
|
||||
"duckdb": "SELECT * FROM test WHERE col1 % 4 = 3",
|
||||
},
|
||||
write={
|
||||
"duckdb": "SELECT * FROM test WHERE col1 % 4 = 3",
|
||||
"oracle": "SELECT * FROM test WHERE MOD(col1, 4) = 3",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CURRENT_TIMESTAMP BETWEEN TO_DATE(f.C_SDATE, 'yyyy/mm/dd') AND TO_DATE(f.C_EDATE, 'yyyy/mm/dd')",
|
||||
read={
|
||||
|
@ -242,6 +252,15 @@ class TestOracle(Validator):
|
|||
"""SELECT * FROM t ORDER BY a ASC, b ASC NULLS FIRST, c DESC NULLS LAST, d DESC""",
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"NVL(NULL, 1)",
|
||||
write={
|
||||
"oracle": "NVL(NULL, 1)",
|
||||
"": "COALESCE(NULL, 1)",
|
||||
"clickhouse": "COALESCE(NULL, 1)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_join_marker(self):
|
||||
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y")
|
||||
|
||||
|
|
|
@ -28,7 +28,7 @@ class TestPostgres(Validator):
|
|||
alter_table_only = """ALTER TABLE ONLY "Album" ADD CONSTRAINT "FK_AlbumArtistId" FOREIGN KEY ("ArtistId") REFERENCES "Artist" ("ArtistId") ON DELETE NO ACTION ON UPDATE NO ACTION"""
|
||||
expr = self.parse_one(alter_table_only)
|
||||
|
||||
self.assertIsInstance(expr, exp.AlterTable)
|
||||
self.assertIsInstance(expr, exp.Alter)
|
||||
self.assertEqual(expr.sql(dialect="postgres"), alter_table_only)
|
||||
|
||||
self.validate_identity("STRING_TO_ARRAY('xx~^~yy~^~zz', '~^~', 'yy')")
|
||||
|
@ -548,47 +548,54 @@ class TestPostgres(Validator):
|
|||
"postgres": "x # y",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT GENERATE_SERIES(1, 5)",
|
||||
write={
|
||||
"bigquery": UnsupportedError,
|
||||
"postgres": "SELECT GENERATE_SERIES(1, 5)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"WITH dates AS (SELECT GENERATE_SERIES('2020-01-01'::DATE, '2024-01-01'::DATE, '1 day'::INTERVAL) AS date), date_table AS (SELECT DISTINCT DATE_TRUNC('MONTH', date) AS date FROM dates) SELECT * FROM date_table",
|
||||
write={
|
||||
"duckdb": "WITH dates AS (SELECT UNNEST(GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2024-01-01' AS DATE), CAST('1 day' AS INTERVAL))) AS date), date_table AS (SELECT DISTINCT DATE_TRUNC('MONTH', date) AS date FROM dates) SELECT * FROM date_table",
|
||||
"postgres": "WITH dates AS (SELECT GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2024-01-01' AS DATE), CAST('1 day' AS INTERVAL)) AS date), date_table AS (SELECT DISTINCT DATE_TRUNC('MONTH', date) AS date FROM dates) SELECT * FROM date_table",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"GENERATE_SERIES(a, b, ' 2 days ')",
|
||||
write={
|
||||
"postgres": "GENERATE_SERIES(a, b, INTERVAL '2 DAYS')",
|
||||
"presto": "SEQUENCE(a, b, INTERVAL '2' DAY)",
|
||||
"trino": "SEQUENCE(a, b, INTERVAL '2' DAY)",
|
||||
"presto": "UNNEST(SEQUENCE(a, b, INTERVAL '2' DAY))",
|
||||
"trino": "UNNEST(SEQUENCE(a, b, INTERVAL '2' DAY))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"GENERATE_SERIES('2019-01-01'::TIMESTAMP, NOW(), '1day')",
|
||||
write={
|
||||
"databricks": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
|
||||
"hive": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
|
||||
"postgres": "GENERATE_SERIES(CAST('2019-01-01' AS TIMESTAMP), CURRENT_TIMESTAMP, INTERVAL '1 DAY')",
|
||||
"presto": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"trino": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"hive": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"spark2": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"spark": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"databricks": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
|
||||
"presto": "UNNEST(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY))",
|
||||
"spark": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
|
||||
"spark2": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
|
||||
"trino": "UNNEST(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"GENERATE_SERIES(a, b)",
|
||||
"SELECT * FROM GENERATE_SERIES(a, b)",
|
||||
read={
|
||||
"postgres": "GENERATE_SERIES(a, b)",
|
||||
"presto": "SEQUENCE(a, b)",
|
||||
"trino": "SEQUENCE(a, b)",
|
||||
"tsql": "GENERATE_SERIES(a, b)",
|
||||
"hive": "SEQUENCE(a, b)",
|
||||
"spark2": "SEQUENCE(a, b)",
|
||||
"spark": "SEQUENCE(a, b)",
|
||||
"databricks": "SEQUENCE(a, b)",
|
||||
"tsql": "SELECT * FROM GENERATE_SERIES(a, b)",
|
||||
},
|
||||
write={
|
||||
"postgres": "GENERATE_SERIES(a, b)",
|
||||
"presto": "SEQUENCE(a, b)",
|
||||
"trino": "SEQUENCE(a, b)",
|
||||
"tsql": "GENERATE_SERIES(a, b)",
|
||||
"hive": "SEQUENCE(a, b)",
|
||||
"spark2": "SEQUENCE(a, b)",
|
||||
"spark": "SEQUENCE(a, b)",
|
||||
"databricks": "SEQUENCE(a, b)",
|
||||
"databricks": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
|
||||
"hive": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
|
||||
"postgres": "SELECT * FROM GENERATE_SERIES(a, b)",
|
||||
"presto": "SELECT * FROM UNNEST(SEQUENCE(a, b))",
|
||||
"spark": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
|
||||
"spark2": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
|
||||
"trino": "SELECT * FROM UNNEST(SEQUENCE(a, b))",
|
||||
"tsql": "SELECT * FROM GENERATE_SERIES(a, b)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
|
@ -13,6 +13,13 @@ class TestPresto(Validator):
|
|||
self.validate_identity("CAST(TDIGEST_AGG(1) AS TDIGEST)")
|
||||
self.validate_identity("CAST(x AS HYPERLOGLOG)")
|
||||
|
||||
self.validate_all(
|
||||
"SELECT FROM_ISO8601_TIMESTAMP('2020-05-11T11:15:05')",
|
||||
write={
|
||||
"duckdb": "SELECT CAST('2020-05-11T11:15:05' AS TIMESTAMPTZ)",
|
||||
"presto": "SELECT FROM_ISO8601_TIMESTAMP('2020-05-11T11:15:05')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CAST(x AS INTERVAL YEAR TO MONTH)",
|
||||
write={
|
||||
|
|
|
@ -6,7 +6,6 @@ class TestRedshift(Validator):
|
|||
dialect = "redshift"
|
||||
|
||||
def test_redshift(self):
|
||||
self.validate_identity("1 div", "1 AS div")
|
||||
self.validate_all(
|
||||
"SELECT SPLIT_TO_ARRAY('12,345,6789')",
|
||||
write={
|
||||
|
@ -315,6 +314,7 @@ class TestRedshift(Validator):
|
|||
)
|
||||
|
||||
def test_identity(self):
|
||||
self.validate_identity("1 div", "1 AS div")
|
||||
self.validate_identity("LISTAGG(DISTINCT foo, ', ')")
|
||||
self.validate_identity("CREATE MATERIALIZED VIEW orders AUTO REFRESH YES AS SELECT 1")
|
||||
self.validate_identity("SELECT DATEADD(DAY, 1, 'today')")
|
||||
|
@ -337,6 +337,10 @@ class TestRedshift(Validator):
|
|||
self.validate_identity(
|
||||
"""SELECT JSON_EXTRACT_PATH_TEXT('{"f2":{"f3":1},"f4":{"f5":99,"f6":"star"}', 'f4', 'f6', TRUE)"""
|
||||
)
|
||||
self.validate_identity(
|
||||
'DATE_PART(year, "somecol")',
|
||||
'EXTRACT(year FROM "somecol")',
|
||||
).this.assert_is(exp.Var)
|
||||
self.validate_identity(
|
||||
"SELECT CONCAT('abc', 'def')",
|
||||
"SELECT 'abc' || 'def'",
|
||||
|
@ -430,6 +434,14 @@ ORDER BY
|
|||
)
|
||||
self.validate_identity("SELECT JSON_PARSE('[]')")
|
||||
|
||||
self.validate_identity("SELECT ARRAY(1, 2, 3)")
|
||||
self.validate_identity("SELECT ARRAY[1, 2, 3]")
|
||||
|
||||
self.validate_identity(
|
||||
"""SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')""",
|
||||
"""SELECT CONVERT_TIMEZONE('UTC', 'America/New_York', '2024-08-06 09:10:00.000')""",
|
||||
)
|
||||
|
||||
def test_values(self):
|
||||
# Test crazy-sized VALUES clause to UNION ALL conversion to ensure we don't get RecursionError
|
||||
values = [str(v) for v in range(0, 10000)]
|
||||
|
@ -608,3 +620,9 @@ FROM (
|
|||
"select a.foo, b.bar, a.baz from a, b where a.baz = b.baz (+)",
|
||||
"SELECT a.foo, b.bar, a.baz FROM a, b WHERE a.baz = b.baz (+)",
|
||||
)
|
||||
|
||||
def test_time(self):
|
||||
self.validate_all(
|
||||
"TIME_TO_STR(a, '%Y-%m-%d %H:%M:%S.%f')",
|
||||
write={"redshift": "TO_CHAR(a, 'YYYY-MM-DD HH24:MI:SS.US')"},
|
||||
)
|
||||
|
|
|
@ -11,6 +11,7 @@ class TestSnowflake(Validator):
|
|||
dialect = "snowflake"
|
||||
|
||||
def test_snowflake(self):
|
||||
self.validate_identity("1 /* /* */")
|
||||
self.validate_identity(
|
||||
"SELECT * FROM table AT (TIMESTAMP => '2024-07-24') UNPIVOT(a FOR b IN (c)) AS pivot_table"
|
||||
)
|
||||
|
@ -59,6 +60,7 @@ WHERE
|
|||
)""",
|
||||
)
|
||||
|
||||
self.validate_identity("exclude := [foo]")
|
||||
self.validate_identity("SELECT CAST([1, 2, 3] AS VECTOR(FLOAT, 3))")
|
||||
self.validate_identity("SELECT CONNECT_BY_ROOT test AS test_column_alias")
|
||||
self.validate_identity("SELECT number").selects[0].assert_is(exp.Column)
|
||||
|
@ -113,6 +115,18 @@ WHERE
|
|||
self.validate_identity("ALTER TABLE a SWAP WITH b")
|
||||
self.validate_identity("SELECT MATCH_CONDITION")
|
||||
self.validate_identity("SELECT * REPLACE (CAST(col AS TEXT) AS scol) FROM t")
|
||||
self.validate_identity(
|
||||
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR quarter IN ('2023_Q1', '2023_Q2', '2023_Q3', '2023_Q4', '2024_Q1') DEFAULT ON NULL (0)) ORDER BY empid"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR quarter IN (SELECT DISTINCT quarter FROM ad_campaign_types_by_quarter WHERE television = TRUE ORDER BY quarter)) ORDER BY empid"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR IN (ANY ORDER BY quarter)) ORDER BY empid"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR IN (ANY)) ORDER BY empid"
|
||||
)
|
||||
self.validate_identity(
|
||||
"MERGE INTO my_db AS ids USING (SELECT new_id FROM my_model WHERE NOT col IS NULL) AS new_ids ON ids.type = new_ids.type AND ids.source = new_ids.source WHEN NOT MATCHED THEN INSERT VALUES (new_ids.new_id)"
|
||||
)
|
||||
|
@ -125,6 +139,18 @@ WHERE
|
|||
self.validate_identity(
|
||||
"SELECT * FROM DATA AS DATA_L ASOF JOIN DATA AS DATA_R MATCH_CONDITION (DATA_L.VAL > DATA_R.VAL) ON DATA_L.ID = DATA_R.ID"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM s WHERE c NOT IN (1, 2, 3)",
|
||||
"SELECT * FROM s WHERE NOT c IN (1, 2, 3)",
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM s WHERE c NOT IN (SELECT * FROM t)",
|
||||
"SELECT * FROM s WHERE c <> ALL (SELECT * FROM t)",
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM t1 INNER JOIN t2 USING (t1.col)",
|
||||
"SELECT * FROM t1 INNER JOIN t2 USING (col)",
|
||||
)
|
||||
self.validate_identity(
|
||||
"CURRENT_TIMESTAMP - INTERVAL '1 w' AND (1 = 1)",
|
||||
"CURRENT_TIMESTAMP() - INTERVAL '1 WEEK' AND (1 = 1)",
|
||||
|
@ -847,6 +873,33 @@ WHERE
|
|||
},
|
||||
)
|
||||
|
||||
self.validate_identity(
|
||||
"""SELECT ARRAY_CONSTRUCT('foo')::VARIANT[0]""",
|
||||
"""SELECT CAST(['foo'] AS VARIANT)[0]""",
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
|
||||
write={
|
||||
"snowflake": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"spark": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"databricks": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"redshift": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
|
||||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
|
||||
write={
|
||||
"snowflake": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"spark": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"databricks": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"redshift": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
|
||||
"mysql": "SELECT CONVERT_TZ('2024-08-06 09:10:00.000', 'America/Los_Angeles', 'America/New_York')",
|
||||
"duckdb": "SELECT CAST('2024-08-06 09:10:00.000' AS TIMESTAMP) AT TIME ZONE 'America/Los_Angeles' AT TIME ZONE 'America/New_York'",
|
||||
},
|
||||
)
|
||||
|
||||
def test_null_treatment(self):
|
||||
self.validate_all(
|
||||
r"SELECT FIRST_VALUE(TABLE1.COLUMN1) OVER (PARTITION BY RANDOM_COLUMN1, RANDOM_COLUMN2 ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS MY_ALIAS FROM TABLE1",
|
||||
|
@ -1921,7 +1974,7 @@ STORAGE_ALLOWED_LOCATIONS=('s3://mybucket1/path1/', 's3://mybucket2/path2/')""",
|
|||
|
||||
def test_swap(self):
|
||||
ast = parse_one("ALTER TABLE a SWAP WITH b", read="snowflake")
|
||||
assert isinstance(ast, exp.AlterTable)
|
||||
assert isinstance(ast, exp.Alter)
|
||||
assert isinstance(ast.args["actions"][0], exp.SwapTable)
|
||||
|
||||
def test_try_cast(self):
|
||||
|
|
|
@ -129,6 +129,16 @@ TBLPROPERTIES (
|
|||
"spark": "ALTER TABLE StudentInfo DROP COLUMNS (LastName, DOB)",
|
||||
},
|
||||
)
|
||||
self.validate_identity("ALTER VIEW StudentInfoView AS SELECT * FROM StudentInfo")
|
||||
self.validate_identity("ALTER VIEW StudentInfoView AS SELECT LastName FROM StudentInfo")
|
||||
self.validate_identity("ALTER VIEW StudentInfoView RENAME TO StudentInfoViewRenamed")
|
||||
self.validate_identity(
|
||||
"ALTER VIEW StudentInfoView SET TBLPROPERTIES ('key1'='val1', 'key2'='val2')"
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER VIEW StudentInfoView UNSET TBLPROPERTIES ('key1', 'key2')",
|
||||
check_command_warning=True,
|
||||
)
|
||||
|
||||
def test_to_date(self):
|
||||
self.validate_all(
|
||||
|
@ -297,6 +307,13 @@ TBLPROPERTIES (
|
|||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT DATE_FORMAT(DATE '2020-01-01', 'EEEE') AS weekday",
|
||||
write={
|
||||
"presto": "SELECT DATE_FORMAT(CAST(CAST('2020-01-01' AS DATE) AS TIMESTAMP), '%W') AS weekday",
|
||||
"spark": "SELECT DATE_FORMAT(CAST(CAST('2020-01-01' AS DATE) AS TIMESTAMP), 'EEEE') AS weekday",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT TRY_ELEMENT_AT(MAP(1, 'a', 2, 'b'), 2)",
|
||||
read={
|
||||
|
@ -557,7 +574,10 @@ TBLPROPERTIES (
|
|||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x AS TIMESTAMP)", read={"trino": "CAST(x AS TIMESTAMP(6) WITH TIME ZONE)"}
|
||||
"CAST(x AS TIMESTAMP)",
|
||||
read={
|
||||
"trino": "CAST(x AS TIMESTAMP(6) WITH TIME ZONE)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT DATE_ADD(my_date_column, 1)",
|
||||
|
@ -688,6 +708,7 @@ TBLPROPERTIES (
|
|||
"trino": "SELECT DATE_ADD('MONTH', 20, col)",
|
||||
},
|
||||
)
|
||||
self.validate_identity("DESCRIBE schema.test PARTITION(ds = '2024-01-01')")
|
||||
|
||||
def test_bool_or(self):
|
||||
self.validate_all(
|
||||
|
@ -805,8 +826,22 @@ TBLPROPERTIES (
|
|||
self.assertEqual(query.sql(name), without_modifiers)
|
||||
|
||||
def test_schema_binding_options(self):
|
||||
for schema_binding in ("BINDING", "COMPENSATION", "TYPE EVOLUTION", "EVOLUTION"):
|
||||
for schema_binding in (
|
||||
"BINDING",
|
||||
"COMPENSATION",
|
||||
"TYPE EVOLUTION",
|
||||
"EVOLUTION",
|
||||
):
|
||||
with self.subTest(f"Test roundtrip of VIEW schema binding {schema_binding}"):
|
||||
self.validate_identity(
|
||||
f"CREATE VIEW emp_v WITH SCHEMA {schema_binding} AS SELECT * FROM emp"
|
||||
)
|
||||
|
||||
def test_minus(self):
|
||||
self.validate_all(
|
||||
"SELECT * FROM db.table1 MINUS SELECT * FROM db.table2",
|
||||
write={
|
||||
"spark": "SELECT * FROM db.table1 EXCEPT SELECT * FROM db.table2",
|
||||
"databricks": "SELECT * FROM db.table1 EXCEPT SELECT * FROM db.table2",
|
||||
},
|
||||
)
|
||||
|
|
|
@ -1,3 +1,4 @@
|
|||
from sqlglot import exp
|
||||
from tests.dialects.test_dialect import Validator
|
||||
|
||||
|
||||
|
@ -31,6 +32,10 @@ class TestTeradata(Validator):
|
|||
},
|
||||
)
|
||||
|
||||
self.validate_identity(
|
||||
"RENAME TABLE emp TO employee", check_command_warning=True
|
||||
).assert_is(exp.Command)
|
||||
|
||||
def test_translate(self):
|
||||
self.validate_all(
|
||||
"TRANSLATE(x USING LATIN_TO_UNICODE)",
|
||||
|
|
|
@ -16,3 +16,35 @@ class TestTrino(Validator):
|
|||
"SELECT TRIM('!foo!', '!')",
|
||||
"SELECT TRIM('!' FROM '!foo!')",
|
||||
)
|
||||
|
||||
def test_ddl(self):
|
||||
self.validate_identity("ALTER TABLE users RENAME TO people")
|
||||
self.validate_identity("ALTER TABLE IF EXISTS users RENAME TO people")
|
||||
self.validate_identity("ALTER TABLE users ADD COLUMN zip VARCHAR")
|
||||
self.validate_identity("ALTER TABLE IF EXISTS users ADD COLUMN IF NOT EXISTS zip VARCHAR")
|
||||
self.validate_identity("ALTER TABLE users DROP COLUMN zip")
|
||||
self.validate_identity("ALTER TABLE IF EXISTS users DROP COLUMN IF EXISTS zip")
|
||||
self.validate_identity("ALTER TABLE users RENAME COLUMN id TO user_id")
|
||||
self.validate_identity("ALTER TABLE IF EXISTS users RENAME COLUMN IF EXISTS id TO user_id")
|
||||
self.validate_identity("ALTER TABLE users ALTER COLUMN id SET DATA TYPE BIGINT")
|
||||
self.validate_identity("ALTER TABLE users ALTER COLUMN id DROP NOT NULL")
|
||||
self.validate_identity(
|
||||
"ALTER TABLE people SET AUTHORIZATION alice", check_command_warning=True
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER TABLE people SET AUTHORIZATION ROLE PUBLIC", check_command_warning=True
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER TABLE people SET PROPERTIES x = 'y'", check_command_warning=True
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER TABLE people SET PROPERTIES foo = 123, 'foo bar' = 456",
|
||||
check_command_warning=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER TABLE people SET PROPERTIES x = DEFAULT", check_command_warning=True
|
||||
)
|
||||
self.validate_identity("ALTER VIEW people RENAME TO users")
|
||||
self.validate_identity(
|
||||
"ALTER VIEW people SET AUTHORIZATION alice", check_command_warning=True
|
||||
)
|
||||
|
|
|
@ -792,7 +792,7 @@ class TestTSQL(Validator):
|
|||
self.validate_identity(f"CREATE VIEW a.b WITH {view_attr} AS SELECT * FROM x")
|
||||
|
||||
self.validate_identity("ALTER TABLE dbo.DocExe DROP CONSTRAINT FK_Column_B").assert_is(
|
||||
exp.AlterTable
|
||||
exp.Alter
|
||||
).args["actions"][0].assert_is(exp.Drop)
|
||||
|
||||
for clustered_keyword in ("CLUSTERED", "NONCLUSTERED"):
|
||||
|
@ -822,6 +822,20 @@ class TestTSQL(Validator):
|
|||
self.validate_identity("ALTER TABLE tbl SET DATA_DELETION=ON")
|
||||
self.validate_identity("ALTER TABLE tbl SET DATA_DELETION=OFF")
|
||||
|
||||
self.validate_identity("ALTER VIEW v AS SELECT a, b, c, d FROM foo")
|
||||
self.validate_identity("ALTER VIEW v AS SELECT * FROM foo WHERE c > 100")
|
||||
self.validate_identity(
|
||||
"ALTER VIEW v WITH SCHEMABINDING AS SELECT * FROM foo WHERE c > 100",
|
||||
check_command_warning=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER VIEW v WITH ENCRYPTION AS SELECT * FROM foo WHERE c > 100",
|
||||
check_command_warning=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"ALTER VIEW v WITH VIEW_METADATA AS SELECT * FROM foo WHERE c > 100",
|
||||
check_command_warning=True,
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < CURRENT_TIMESTAMP - 7 END",
|
||||
"CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < GETDATE() - 7 END",
|
||||
|
@ -1513,6 +1527,15 @@ WHERE
|
|||
},
|
||||
)
|
||||
|
||||
# Check superfluous casts arent added. ref: https://github.com/TobikoData/sqlmesh/issues/2672
|
||||
self.validate_all(
|
||||
"SELECT DATEDIFF(DAY, CAST(a AS DATETIME2), CAST(b AS DATETIME2)) AS x FROM foo",
|
||||
write={
|
||||
"tsql": "SELECT DATEDIFF(DAY, CAST(a AS DATETIME2), CAST(b AS DATETIME2)) AS x FROM foo",
|
||||
"clickhouse": "SELECT DATE_DIFF(DAY, CAST(a AS Nullable(DateTime)), CAST(b AS Nullable(DateTime))) AS x FROM foo",
|
||||
},
|
||||
)
|
||||
|
||||
def test_lateral_subquery(self):
|
||||
self.validate_all(
|
||||
"SELECT x.a, x.b, t.v, t.y FROM x CROSS APPLY (SELECT v, y FROM t) t(v, y)",
|
||||
|
@ -1650,7 +1673,7 @@ WHERE
|
|||
},
|
||||
write={
|
||||
"bigquery": "LAST_DAY(CAST(CURRENT_TIMESTAMP() AS DATE))",
|
||||
"clickhouse": "LAST_DAY(CAST(CURRENT_TIMESTAMP() AS DATE))",
|
||||
"clickhouse": "LAST_DAY(CAST(CURRENT_TIMESTAMP() AS Nullable(DATE)))",
|
||||
"duckdb": "LAST_DAY(CAST(CURRENT_TIMESTAMP AS DATE))",
|
||||
"mysql": "LAST_DAY(DATE(CURRENT_TIMESTAMP()))",
|
||||
"postgres": "CAST(DATE_TRUNC('MONTH', CAST(CURRENT_TIMESTAMP AS DATE)) + INTERVAL '1 MONTH' - INTERVAL '1 DAY' AS DATE)",
|
||||
|
@ -1665,7 +1688,7 @@ WHERE
|
|||
"EOMONTH(GETDATE(), -1)",
|
||||
write={
|
||||
"bigquery": "LAST_DAY(DATE_ADD(CAST(CURRENT_TIMESTAMP() AS DATE), INTERVAL -1 MONTH))",
|
||||
"clickhouse": "LAST_DAY(DATE_ADD(MONTH, -1, CAST(CURRENT_TIMESTAMP() AS DATE)))",
|
||||
"clickhouse": "LAST_DAY(DATE_ADD(MONTH, -1, CAST(CURRENT_TIMESTAMP() AS Nullable(DATE))))",
|
||||
"duckdb": "LAST_DAY(CAST(CURRENT_TIMESTAMP AS DATE) + INTERVAL (-1) MONTH)",
|
||||
"mysql": "LAST_DAY(DATE_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 MONTH))",
|
||||
"postgres": "CAST(DATE_TRUNC('MONTH', CAST(CURRENT_TIMESTAMP AS DATE) + INTERVAL '-1 MONTH') + INTERVAL '1 MONTH' - INTERVAL '1 DAY' AS DATE)",
|
||||
|
|
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue