dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Adding upstream version 20.1.0.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-13 21:16:46 +01:00
parent 6a89523da4
commit 5bd573dda1
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
127 changed files with 73384 additions and 73067 deletions
Download patch file Download diff file Expand all files Collapse all files

364
CHANGELOG.md
View file

@ -1,6 +1,349 @@
Changelog Changelog
========= =========
## [v20.0.0] - 2023-12-07
### :boom: BREAKING CHANGES
- due to [`be89da3`](https://github.com/tobymao/sqlglot/commit/be89da3747fa95d98e0d5a28d19d98f5822a8979) - introduce Dialect settings, make MySQL case-sensitive by default *(PR [#2627](https://github.com/tobymao/sqlglot/pull/2627) by [@georgesittas](https://github.com/georgesittas))*:
introduce Dialect settings, make MySQL case-sensitive by default (#2627)
- due to [`4d68e39`](https://github.com/tobymao/sqlglot/commit/4d68e39a3aadc9b07d3f96c51fc46fd407486d86) - remove redundant todate closes [#2636](https://github.com/tobymao/sqlglot/pull/2636) *(commit by [@tobymao](https://github.com/tobymao))*:
remove redundant todate closes #2636
- due to [`1e387f6`](https://github.com/tobymao/sqlglot/commit/1e387f63853efbcc4da7ce8d822fe8975ffe52f3) - parse functions with positional args in exp.func *(PR [#2622](https://github.com/tobymao/sqlglot/pull/2622) by [@georgesittas](https://github.com/georgesittas))*:
parse functions with positional args in exp.func (#2622)
- due to [`ee2e7f0`](https://github.com/tobymao/sqlglot/commit/ee2e7f099b46d2c8548ab34784d19b77fe838ce7) - snowflake column transform constraints closes [#2634](https://github.com/tobymao/sqlglot/pull/2634) *(commit by [@tobymao](https://github.com/tobymao))*:
snowflake column transform constraints closes #2634
- due to [`656d54c`](https://github.com/tobymao/sqlglot/commit/656d54c808d65a2ec1443719e3f35dc651ed98ab) - make lineage html more reusable *(commit by [@tobymao](https://github.com/tobymao))*:
make lineage html more reusable
### :sparkles: New Features
- [`be89da3`](https://github.com/tobymao/sqlglot/commit/be89da3747fa95d98e0d5a28d19d98f5822a8979) - introduce Dialect settings, make MySQL case-sensitive by default *(PR [#2627](https://github.com/tobymao/sqlglot/pull/2627) by [@georgesittas](https://github.com/georgesittas))*
- [`ee2e7f0`](https://github.com/tobymao/sqlglot/commit/ee2e7f099b46d2c8548ab34784d19b77fe838ce7) - snowflake column transform constraints closes [#2634](https://github.com/tobymao/sqlglot/pull/2634) *(commit by [@tobymao](https://github.com/tobymao))*
### :bug: Bug Fixes
- [`b0c5375`](https://github.com/tobymao/sqlglot/commit/b0c5375be98ac7e74870ea69d58c211cf9c73dea) - **tsql**: add dw, hour to the DATEPART-only formats *(PR [#2632](https://github.com/tobymao/sqlglot/pull/2632) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2630](undefined) opened by [@abadugu13](https://github.com/abadugu13)*
- [`4d68e39`](https://github.com/tobymao/sqlglot/commit/4d68e39a3aadc9b07d3f96c51fc46fd407486d86) - remove redundant todate closes [#2636](https://github.com/tobymao/sqlglot/pull/2636) *(commit by [@tobymao](https://github.com/tobymao))*
- [`1e387f6`](https://github.com/tobymao/sqlglot/commit/1e387f63853efbcc4da7ce8d822fe8975ffe52f3) - parse functions with positional args in exp.func *(PR [#2622](https://github.com/tobymao/sqlglot/pull/2622) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2621](undefined) opened by [@cpcloud](https://github.com/cpcloud)*
- :arrow_lower_right: *fixes issue [#2631](undefined) opened by [@cpcloud](https://github.com/cpcloud)*
- [`78697b4`](https://github.com/tobymao/sqlglot/commit/78697b48d9cf310b9098aa48c5180acf279d9945) - **optimizer**: simplify Sub/Div more conservatively, they're not associative *(PR [#2635](https://github.com/tobymao/sqlglot/pull/2635) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2633](undefined) opened by [@jamespan](https://github.com/jamespan)*
### :recycle: Refactors
- [`656d54c`](https://github.com/tobymao/sqlglot/commit/656d54c808d65a2ec1443719e3f35dc651ed98ab) - make lineage html more reusable *(commit by [@tobymao](https://github.com/tobymao))*
### :wrench: Chores
- [`12a00e9`](https://github.com/tobymao/sqlglot/commit/12a00e985dacf508f8268b2bb209156f748e197c) - make normalization strategy str enum *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.9.0] - 2023-12-05
### :boom: BREAKING CHANGES
- due to [`6e71c34`](https://github.com/tobymao/sqlglot/commit/6e71c348eab63125e412382381acbcbd79efac43) - remove safe versions and use a flag instead *(PR [#2629](https://github.com/tobymao/sqlglot/pull/2629) by [@tobymao](https://github.com/tobymao))*:
remove safe versions and use a flag instead (#2629)
### :bug: Bug Fixes
- [`4755293`](https://github.com/tobymao/sqlglot/commit/4755293dba1a82df7d6a520056ecb81c6a5117ea) - attach function comments to the AST *(PR [#2628](https://github.com/tobymao/sqlglot/pull/2628) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2626](undefined) opened by [@aersam](https://github.com/aersam)*
- [`160f06d`](https://github.com/tobymao/sqlglot/commit/160f06dc902cc9c666093c47a811d7bd18c91a30) - **snowflake**: allow rename/replace identifier *(commit by [@tobymao](https://github.com/tobymao))*
### :recycle: Refactors
- [`6e71c34`](https://github.com/tobymao/sqlglot/commit/6e71c348eab63125e412382381acbcbd79efac43) - remove safe versions and use a flag instead *(PR [#2629](https://github.com/tobymao/sqlglot/pull/2629) by [@tobymao](https://github.com/tobymao))*
## [v19.8.3] - 2023-12-04
### :sparkles: New Features
- [`05b41e8`](https://github.com/tobymao/sqlglot/commit/05b41e81639eabe6f2c4116fb7c14ea99da8b655) - add identify to table_name *(commit by [@tobymao](https://github.com/tobymao))*
### :bug: Bug Fixes
- [`6351007`](https://github.com/tobymao/sqlglot/commit/6351007e36bf7af9dfdb0b797bed4834fa394de1) - tokenize CRLF sequence correctly *(PR [#2623](https://github.com/tobymao/sqlglot/pull/2623) by [@georgesittas](https://github.com/georgesittas))*
### :wrench: Chores
- [`f9a43a1`](https://github.com/tobymao/sqlglot/commit/f9a43a176fcd6ab952bfd8bca21473d541423156) - don't patch loggers at method level to silence warnings *(PR [#2620](https://github.com/tobymao/sqlglot/pull/2620) by [@georgesittas](https://github.com/georgesittas))*
## [v19.8.2] - 2023-12-01
### :bug: Bug Fixes
- [`5657a60`](https://github.com/tobymao/sqlglot/commit/5657a60169680c45feb67fe7a1da16b4c9ee22b6) - **tsql, teradata**: Distinct goes before top *(PR [#2618](https://github.com/tobymao/sqlglot/pull/2618) by [@treysp](https://github.com/treysp))*
- [`c0e751a`](https://github.com/tobymao/sqlglot/commit/c0e751a71cd69746b7acb5f1950f1b925c826373) - **duckdb**: arrays are 1-indexed *(PR [#2619](https://github.com/tobymao/sqlglot/pull/2619) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2617](undefined) opened by [@j1ah0ng](https://github.com/j1ah0ng)*
## [v19.8.1] - 2023-12-01
### :bug: Bug Fixes
- [`441f624`](https://github.com/tobymao/sqlglot/commit/441f624c5cafb49e3d73d4435b7cf4bf3e891150) - cannot have union with limit *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.8.0] - 2023-12-01
### :boom: BREAKING CHANGES
- due to [`b5f690b`](https://github.com/tobymao/sqlglot/commit/b5f690bc36e2278ec9d9299041497485f73198a8) - add timestamp functions to BQ and DuckDB closes [#2611](https://github.com/tobymao/sqlglot/pull/2611) *(PR [#2612](https://github.com/tobymao/sqlglot/pull/2612) by [@j1ah0ng](https://github.com/j1ah0ng))*:
add timestamp functions to BQ and DuckDB closes #2611 (#2612)
- due to [`019e0e5`](https://github.com/tobymao/sqlglot/commit/019e0e5ba4b5df1ef3b34510c2fa07f8623af364) - qualify columns added in explode to unnest transformation *(PR [#2615](https://github.com/tobymao/sqlglot/pull/2615) by [@georgesittas](https://github.com/georgesittas))*:
qualify columns added in explode to unnest transformation (#2615)
### :sparkles: New Features
- [`5af7ac3`](https://github.com/tobymao/sqlglot/commit/5af7ac359efc7d2575eb2cdfd0fe34b9518805c2) - helper method for dot parts *(commit by [@tobymao](https://github.com/tobymao))*
- [`da0a4b1`](https://github.com/tobymao/sqlglot/commit/da0a4b1cc3d093012e4a92a9bb6f70c7db11749c) - **postgres**: add support for operators with schema path *(PR [#2610](https://github.com/tobymao/sqlglot/pull/2610) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#2609](undefined) opened by [@ninja96826](https://github.com/ninja96826)*
### :bug: Bug Fixes
- [`568ddd1`](https://github.com/tobymao/sqlglot/commit/568ddd12d98142146073e27e206426a66b73eb42) - treat parameters as primary expressions *(PR [#2605](https://github.com/tobymao/sqlglot/pull/2605) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2604](undefined) opened by [@bruderooo](https://github.com/bruderooo)*
- [`526d760`](https://github.com/tobymao/sqlglot/commit/526d7602018268f8dc6e8111b26de4df8601d0cf) - revert 568ddd12, parse placeholder in sample instead *(PR [#2606](https://github.com/tobymao/sqlglot/pull/2606) by [@georgesittas](https://github.com/georgesittas))*
- [`bb75218`](https://github.com/tobymao/sqlglot/commit/bb7521820d6d22a669bfb8cbbc0ee8a37d32c361) - always expand sentinel line break in pretty mode *(PR [#2608](https://github.com/tobymao/sqlglot/pull/2608) by [@georgesittas](https://github.com/georgesittas))*
- [`9106702`](https://github.com/tobymao/sqlglot/commit/9106702decda3957ef0f88f4a4d88b5a14a55a8c) - properly normalize and parse schema for replace_tables and expand *(commit by [@tobymao](https://github.com/tobymao))*
- [`d47879f`](https://github.com/tobymao/sqlglot/commit/d47879f049914ec94df8b09cd5a60e8ad64b2f59) - **snowflake**: unnest sql doesn't need subquery *(commit by [@tobymao](https://github.com/tobymao))*
- [`b5f690b`](https://github.com/tobymao/sqlglot/commit/b5f690bc36e2278ec9d9299041497485f73198a8) - add timestamp functions to BQ and DuckDB closes [#2611](https://github.com/tobymao/sqlglot/pull/2611) *(PR [#2612](https://github.com/tobymao/sqlglot/pull/2612) by [@j1ah0ng](https://github.com/j1ah0ng))*
- [`5aa134d`](https://github.com/tobymao/sqlglot/commit/5aa134d8489cea96d8fa891aaa26e68983ee7537) - preserve alias quotes in explode_to_unnest *(PR [#2613](https://github.com/tobymao/sqlglot/pull/2613) by [@georgesittas](https://github.com/georgesittas))*
- [`5509e31`](https://github.com/tobymao/sqlglot/commit/5509e31cde2e008e6afbe352d4700acb1d6b9c25) - generate UnixToTime correctly (spark, bq, presto, snowflake, duckdb) *(PR [#2614](https://github.com/tobymao/sqlglot/pull/2614) by [@georgesittas](https://github.com/georgesittas))*
- [`019e0e5`](https://github.com/tobymao/sqlglot/commit/019e0e5ba4b5df1ef3b34510c2fa07f8623af364) - qualify columns added in explode to unnest transformation *(PR [#2615](https://github.com/tobymao/sqlglot/pull/2615) by [@georgesittas](https://github.com/georgesittas))*
- [`ad9fe11`](https://github.com/tobymao/sqlglot/commit/ad9fe1156c55d6aa278a43bc979da216c2a1a7d9) - **snowflake**: snowflake array_contains closes [#2616](https://github.com/tobymao/sqlglot/pull/2616) *(commit by [@tobymao](https://github.com/tobymao))*
### :wrench: Chores
- [`4ec01d3`](https://github.com/tobymao/sqlglot/commit/4ec01d398535738a55c15202a5a88bae3f9a86dc) - cleanup types *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.7.0] - 2023-11-28
### :boom: BREAKING CHANGES
- due to [`8cd7d1c`](https://github.com/tobymao/sqlglot/commit/8cd7d1c0bb56aff6bcd08a3ae4e71f68022307b8) - use more canonical cast instead of to_date *(commit by [@tobymao](https://github.com/tobymao))*:
use more canonical cast instead of to_date
- due to [`c413b7f`](https://github.com/tobymao/sqlglot/commit/c413b7fa56c69da3297eb575de7600316b491f18) - expand positional args in order by as aliases *(PR [#2599](https://github.com/tobymao/sqlglot/pull/2599) by [@tobymao](https://github.com/tobymao))*:
expand positional args in order by as aliases (#2599)
- due to [`13817f1`](https://github.com/tobymao/sqlglot/commit/13817f187a79bcc559f7e6939729fce8fecbd812) - avoid unnecessary copying in normalization *(PR [#2602](https://github.com/tobymao/sqlglot/pull/2602) by [@tobymao](https://github.com/tobymao))*:
avoid unnecessary copying in normalization (#2602)
### :sparkles: New Features
- [`739c3c7`](https://github.com/tobymao/sqlglot/commit/739c3c7e4ab1c5986323f2ff6e0ad24bfab8fa0a) - insert returning builder closes [#2579](https://github.com/tobymao/sqlglot/pull/2579) *(commit by [@tobymao](https://github.com/tobymao))*
- [`6e3c7c1`](https://github.com/tobymao/sqlglot/commit/6e3c7c197e8ca2dd48af389e12951ddd22313430) - **tsql**: default database .. closes [#2594](https://github.com/tobymao/sqlglot/pull/2594) *(commit by [@tobymao](https://github.com/tobymao))*
### :bug: Bug Fixes
- [`8cd7d1c`](https://github.com/tobymao/sqlglot/commit/8cd7d1c0bb56aff6bcd08a3ae4e71f68022307b8) - use more canonical cast instead of to_date *(commit by [@tobymao](https://github.com/tobymao))*
- [`08d60b6`](https://github.com/tobymao/sqlglot/commit/08d60b6ef414515920666b56ee8fadb386df0022) - **tsql**: add special chars in single var tokens *(PR [#2582](https://github.com/tobymao/sqlglot/pull/2582) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2581](undefined) opened by [@Hal-H2Apps](https://github.com/Hal-H2Apps)*
- [`bf29a9b`](https://github.com/tobymao/sqlglot/commit/bf29a9b04152672d261649f392316cdaa39ef1e2) - handle ending spaces after keywords closes [#2585](https://github.com/tobymao/sqlglot/pull/2585) *(commit by [@tobymao](https://github.com/tobymao))*
- [`f53f656`](https://github.com/tobymao/sqlglot/commit/f53f6565330cf594c07977b4f1d169220ae8fe19) - **optimizer**: respect EXCEPT when expanding star for PIVOTs *(PR [#2589](https://github.com/tobymao/sqlglot/pull/2589) by [@georgesittas](https://github.com/georgesittas))*
- [`426075f`](https://github.com/tobymao/sqlglot/commit/426075fe17b60b419f38a8ef5735977b953b92af) - **duckdb**: unqualify columns under Pivot *(PR [#2590](https://github.com/tobymao/sqlglot/pull/2590) by [@georgesittas](https://github.com/georgesittas))*
- [`4774431`](https://github.com/tobymao/sqlglot/commit/4774431e7f9cf8146d0e4721d8c49957696c7727) - **tsql**: generate DATEPART when the format is quarter *(PR [#2591](https://github.com/tobymao/sqlglot/pull/2591) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2586](undefined) opened by [@abadugu13](https://github.com/abadugu13)*
- [`dc783a8`](https://github.com/tobymao/sqlglot/commit/dc783a8b723ca000e6ff2343675f4f0030716037) - **bigquery**: generate FORMAT_DATE for TimeToStr *(PR [#2596](https://github.com/tobymao/sqlglot/pull/2596) by [@georgesittas](https://github.com/georgesittas))*
- [`2ecfd34`](https://github.com/tobymao/sqlglot/commit/2ecfd34628853711dc30e86b3888a88a2d0869a0) - time format chunk misses from mapping, but its constituent parts are not *(PR [#2598](https://github.com/tobymao/sqlglot/pull/2598) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2597](undefined) opened by [@j-bennet](https://github.com/j-bennet)*
- [`c413b7f`](https://github.com/tobymao/sqlglot/commit/c413b7fa56c69da3297eb575de7600316b491f18) - expand positional args in order by as aliases *(PR [#2599](https://github.com/tobymao/sqlglot/pull/2599) by [@tobymao](https://github.com/tobymao))*
- [`c66e413`](https://github.com/tobymao/sqlglot/commit/c66e413c916c15d8e485d42055a9214db936c56c) - **oracle**: to_char nlsparam closes [#2601](https://github.com/tobymao/sqlglot/pull/2601) *(commit by [@tobymao](https://github.com/tobymao))*
### :recycle: Refactors
- [`13817f1`](https://github.com/tobymao/sqlglot/commit/13817f187a79bcc559f7e6939729fce8fecbd812) - avoid unnecessary copying in normalization *(PR [#2602](https://github.com/tobymao/sqlglot/pull/2602) by [@tobymao](https://github.com/tobymao))*
## [v19.6.0] - 2023-11-20
### :bug: Bug Fixes
- [`7647227`](https://github.com/tobymao/sqlglot/commit/76472275018a069abc7079bc52cac94fb8d2d348) - **oracle**: parse DROP CONSTRAINT into DROP instead of Command *(PR [#2573](https://github.com/tobymao/sqlglot/pull/2573) by [@HassanShafiq123](https://github.com/HassanShafiq123))*
- :arrow_lower_right: *fixes issue [#2572](undefined) opened by [@HassanShafiq123](https://github.com/HassanShafiq123)*
- [`f5899a1`](https://github.com/tobymao/sqlglot/commit/f5899a1a0da096e012b7abd0627a372e4202a612) - **bigquery**: bigquery only allows literals in LIMIT *(PR [#2574](https://github.com/tobymao/sqlglot/pull/2574) by [@treysp](https://github.com/treysp))*
- [`757c433`](https://github.com/tobymao/sqlglot/commit/757c433944d6afd942b74edc84afb07b140e1a1c) - treat := as PropertyEQ in base sqlglot classes *(PR [#2576](https://github.com/tobymao/sqlglot/pull/2576) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2575](undefined) opened by [@j-bennet](https://github.com/j-bennet)*
### :wrench: Chores
- [`1e16001`](https://github.com/tobymao/sqlglot/commit/1e160012aecfb754294e2d6e207610741bdd264a) - cleanup tests *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.5.1] - 2023-11-16
### :bug: Bug Fixes
- [`932b610`](https://github.com/tobymao/sqlglot/commit/932b610bae7996b488a0406db76f308ddbfd5e44) - interval simplification with non literal *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.5.0] - 2023-11-16
### :boom: BREAKING CHANGES
- due to [`96d514c`](https://github.com/tobymao/sqlglot/commit/96d514c52e9f467d0fac196cee3c7ba3441ace5b) - get rid of SetAgg to use ArrayUniqueAgg for consistency *(PR [#2566](https://github.com/tobymao/sqlglot/pull/2566) by [@georgesittas](https://github.com/georgesittas))*:
get rid of SetAgg to use ArrayUniqueAgg for consistency (#2566)
### :sparkles: New Features
- [`3766686`](https://github.com/tobymao/sqlglot/commit/3766686999cb7f80cdca46150d7e74f954d84cbd) - **executor**: add support for array_unique_agg *(PR [#2564](https://github.com/tobymao/sqlglot/pull/2564) by [@wezham](https://github.com/wezham))*
- [`53b3677`](https://github.com/tobymao/sqlglot/commit/53b3677c919811424f3d0ed5f21aa9c0e76452f5) - **executor**: add support for null replacement value in ARRAY_JOIN *(PR [#2569](https://github.com/tobymao/sqlglot/pull/2569) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`ac79a59`](https://github.com/tobymao/sqlglot/commit/ac79a59a0b2a6143689427a4c93a6548b19ef5f8) - tokenizer should not reuse list in initializer *(commit by [@tobymao](https://github.com/tobymao))*
- [`2029896`](https://github.com/tobymao/sqlglot/commit/2029896d672b021d9fb471162c1933bfdd2c10d1) - **snowflake**: Snowflake only supports literals in LIMIT *(PR [#2568](https://github.com/tobymao/sqlglot/pull/2568) by [@treysp](https://github.com/treysp))*
### :recycle: Refactors
- [`96d514c`](https://github.com/tobymao/sqlglot/commit/96d514c52e9f467d0fac196cee3c7ba3441ace5b) - get rid of SetAgg to use ArrayUniqueAgg for consistency *(PR [#2566](https://github.com/tobymao/sqlglot/pull/2566) by [@georgesittas](https://github.com/georgesittas))*
- [`c4da9fc`](https://github.com/tobymao/sqlglot/commit/c4da9fc9b657a90791d4189d551d5963742f6188) - tokenizer performance improvements *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.4.0] - 2023-11-14
### :boom: BREAKING CHANGES
- due to [`5034d92`](https://github.com/tobymao/sqlglot/commit/5034d92d63aaf4d6861a7c6bf2c041c60fb8043c) - transpile NULLS FIRST/LAST to dialects that dont support it *(PR [#2554](https://github.com/tobymao/sqlglot/pull/2554) by [@georgesittas](https://github.com/georgesittas))*:
transpile NULLS FIRST/LAST to dialects that dont support it (#2554)
### :sparkles: New Features
- [`5034d92`](https://github.com/tobymao/sqlglot/commit/5034d92d63aaf4d6861a7c6bf2c041c60fb8043c) - transpile NULLS FIRST/LAST to dialects that dont support it *(PR [#2554](https://github.com/tobymao/sqlglot/pull/2554) by [@georgesittas](https://github.com/georgesittas))*
- [`a6bdff9`](https://github.com/tobymao/sqlglot/commit/a6bdff945860f758222df4b0b6051542522fd697) - **snowflake**: improve TRY_CAST -> CAST transpilation *(PR [#2561](https://github.com/tobymao/sqlglot/pull/2561) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`9d345e5`](https://github.com/tobymao/sqlglot/commit/9d345e586354cf40e973e0980535f9a0e5698f3e) - **spark**: string(n) -> varchar(n) closes [#2552](https://github.com/tobymao/sqlglot/pull/2552) *(commit by [@tobymao](https://github.com/tobymao))*
- [`e5e6d92`](https://github.com/tobymao/sqlglot/commit/e5e6d9242de733c95da5a179a2a5acf824f47476) - **optimizer**: dont cast right side of IS *(PR [#2559](https://github.com/tobymao/sqlglot/pull/2559) by [@barakalon](https://github.com/barakalon))*
- [`18793b0`](https://github.com/tobymao/sqlglot/commit/18793b05be1ffbe9e42bfd1e274479d122067880) - **snowflake**: generate SHA1 for exp.SHA *(PR [#2557](https://github.com/tobymao/sqlglot/pull/2557) by [@georgesittas](https://github.com/georgesittas))*
- [`8cfb39e`](https://github.com/tobymao/sqlglot/commit/8cfb39eeccfec3a246d1e28c1922d7f605ec53f5) - **clickhouse**: scalar ctes second try *(commit by [@tobymao](https://github.com/tobymao))*
- [`0f9912f`](https://github.com/tobymao/sqlglot/commit/0f9912f03fb5ae99bfa9abfeb44a6e975d88600a) - **snowflake**: only generate TRY_CAST if cast value is of text type *(PR [#2560](https://github.com/tobymao/sqlglot/pull/2560) by [@georgesittas](https://github.com/georgesittas))*
## [v19.3.1] - 2023-11-10
### :bug: Bug Fixes
- [`1d557a7`](https://github.com/tobymao/sqlglot/commit/1d557a76496fd73552782a3f48d70253060faf77) - **tsql**: only call subquery method in CTAS if it's not one already *(PR [#2553](https://github.com/tobymao/sqlglot/pull/2553) by [@georgesittas](https://github.com/georgesittas))*
## [v19.3.0] - 2023-11-10
### :sparkles: New Features
- [`1136f86`](https://github.com/tobymao/sqlglot/commit/1136f866ff491f265e7a4fd75d8e1efe3c300b33) - add exp.IsInf expression to represent ISINF, IS_INF *(PR [#2548](https://github.com/tobymao/sqlglot/pull/2548) by [@j1ah0ng](https://github.com/j1ah0ng))*
### :bug: Bug Fixes
- [`d92f2be`](https://github.com/tobymao/sqlglot/commit/d92f2beeeddf1f8e752e412e89212865e13e2128) - don't bubble up CTEs for the CREATE DDL statement *(PR [#2550](https://github.com/tobymao/sqlglot/pull/2550) by [@georgesittas](https://github.com/georgesittas))*
- [`39ef0e1`](https://github.com/tobymao/sqlglot/commit/39ef0e131670c835a38619d3a6eb4be7a0680881) - **tsql**: preserve column projection quotes for newly added Alias nodes *(PR [#2551](https://github.com/tobymao/sqlglot/pull/2551) by [@georgesittas](https://github.com/georgesittas))*
## [v19.2.0] - 2023-11-10
### :boom: BREAKING CHANGES
- due to [`9f42b6b`](https://github.com/tobymao/sqlglot/commit/9f42b6b0b49a7ef6ce7147347fbb136b51d2e8a1) - disallow nested CTEs for Spark and Databricks *(PR [#2544](https://github.com/tobymao/sqlglot/pull/2544) by [@georgesittas](https://github.com/georgesittas))*:
disallow nested CTEs for Spark and Databricks (#2544)
### :sparkles: New Features
- [`91483b0`](https://github.com/tobymao/sqlglot/commit/91483b0daef2aec4a076b6d4214983d14bf9f105) - add / move fixed-width integer tokens to base class *(PR [#2540](https://github.com/tobymao/sqlglot/pull/2540) by [@j1ah0ng](https://github.com/j1ah0ng))*
- [`45334eb`](https://github.com/tobymao/sqlglot/commit/45334eb226199b8e48ab2751567f65e8675f4ff5) - **bigquery**: array contains to exist unnest closes [#2547](https://github.com/tobymao/sqlglot/pull/2547) *(commit by [@tobymao](https://github.com/tobymao))*
### :bug: Bug Fixes
- [`9f42b6b`](https://github.com/tobymao/sqlglot/commit/9f42b6b0b49a7ef6ce7147347fbb136b51d2e8a1) - disallow nested CTEs for Spark and Databricks *(PR [#2544](https://github.com/tobymao/sqlglot/pull/2544) by [@georgesittas](https://github.com/georgesittas))*
## [v19.1.3] - 2023-11-09
### :sparkles: New Features
- [`d9d64e0`](https://github.com/tobymao/sqlglot/commit/d9d64e0d82c4c89dca132c4d16c0024fefd7aeda) - **postgres**: parse CREATE CONSTRAINT TRIGGER as Command *(PR [#2541](https://github.com/tobymao/sqlglot/pull/2541) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#2538](undefined) opened by [@sweigert](https://github.com/sweigert)*
- [`ea41ddc`](https://github.com/tobymao/sqlglot/commit/ea41ddc69ff05aaf9c54bd0b5a1bea8ffbbf1c5b) - **optimizer**: simplify DATE_ADD on literals *(PR [#2537](https://github.com/tobymao/sqlglot/pull/2537) by [@barakalon](https://github.com/barakalon))*
- [`73746ed`](https://github.com/tobymao/sqlglot/commit/73746edf5c34851a5ab27569cf16885fa121b1db) - more robust boolean conversions for tsql *(PR [#2543](https://github.com/tobymao/sqlglot/pull/2543) by [@tobymao](https://github.com/tobymao))*
### :bug: Bug Fixes
- [`9922232`](https://github.com/tobymao/sqlglot/commit/9922232e4630fea627c882e432c220a18201c0e4) - presto -> spark to_json closes [#2536](https://github.com/tobymao/sqlglot/pull/2536) *(commit by [@tobymao](https://github.com/tobymao))*
- [`1c6d348`](https://github.com/tobymao/sqlglot/commit/1c6d3484ee97c65462e86ef93463a72f5392ae47) - **parser**: take TokenType.RAW_STRING into account in _parse_string *(PR [#2542](https://github.com/tobymao/sqlglot/pull/2542) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2539](undefined) opened by [@braunreyes](https://github.com/braunreyes)*
- [`b5a477f`](https://github.com/tobymao/sqlglot/commit/b5a477f93128ddd816f7f59ca923c07719ae7805) - tsql top paren term parsing *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.1.2] - 2023-11-09
### :sparkles: New Features
- [`7de4922`](https://github.com/tobymao/sqlglot/commit/7de4922e3d111c66d1f6bf25efb1640e41a09383) - **hive**: add fine-grained parsing for REFRESH *(PR [#2531](https://github.com/tobymao/sqlglot/pull/2531) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#2530](undefined) opened by [@juliands-stripe](https://github.com/juliands-stripe)*
### :bug: Bug Fixes
- [`8abf1d7`](https://github.com/tobymao/sqlglot/commit/8abf1d77589af1d7cae7c47ee82ca14833b9966a) - **snowflake**: avoid advancing beyond array limit when parsing staged files *(PR [#2529](https://github.com/tobymao/sqlglot/pull/2529) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2528](undefined) opened by [@nsenno-dbr](https://github.com/nsenno-dbr)*
- [`151f14b`](https://github.com/tobymao/sqlglot/commit/151f14ba66a5f3f37d8159450df5813f82fa4427) - transpile Snowflake structs correctly *(PR [#2534](https://github.com/tobymao/sqlglot/pull/2534) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2533](undefined) opened by [@nsenno-dbr](https://github.com/nsenno-dbr)*
- [`5c750f3`](https://github.com/tobymao/sqlglot/commit/5c750f30896c665b7ea80db0c671af723b39cc93) - **tsql**: convert boolean columns into explicit conditions *(PR [#2535](https://github.com/tobymao/sqlglot/pull/2535) by [@georgesittas](https://github.com/georgesittas))*
## [v19.1.1] - 2023-11-08
### :bug: Bug Fixes
- [`ff69304`](https://github.com/tobymao/sqlglot/commit/ff693048669170b090ad7ab2e6fd76f06b057c2f) - snowflake->spark sample transpilation closes [#2526](https://github.com/tobymao/sqlglot/pull/2526) *(commit by [@tobymao](https://github.com/tobymao))*
- [`a43132b`](https://github.com/tobymao/sqlglot/commit/a43132bfaa4c18e88ad1a0156df020e78f3e0dfe) - Alter column set type statement for MySQL *(PR [#2527](https://github.com/tobymao/sqlglot/pull/2527) by [@izeigerman](https://github.com/izeigerman))*
## [v19.1.0] - 2023-11-08
### :boom: BREAKING CHANGES
- due to [`c6db124`](https://github.com/tobymao/sqlglot/commit/c6db1240b8481af0003a4bed4225f8e46578f182) - transpile division *(PR [#2513](https://github.com/tobymao/sqlglot/pull/2513) by [@barakalon](https://github.com/barakalon))*:
transpile division (#2513)
- due to [`3469e75`](https://github.com/tobymao/sqlglot/commit/3469e75f6fc0c34d68c3a6f1a1b51d8cce3439ff) - typed div and safe div semantics *(PR [#2516](https://github.com/tobymao/sqlglot/pull/2516) by [@barakalon](https://github.com/barakalon))*:
typed div and safe div semantics (#2516)
### :sparkles: New Features
- [`f95947f`](https://github.com/tobymao/sqlglot/commit/f95947f72029a1c19433fa90e4e188af5de28faf) - **bigquery**: add support for FOR .. IN statement *(PR [#2507](https://github.com/tobymao/sqlglot/pull/2507) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#2506](undefined) opened by [@scholtzan](https://github.com/scholtzan)*
- [`01d446b`](https://github.com/tobymao/sqlglot/commit/01d446b7cd9fb73de44feaf5d6665806e3e4a16b) - **optimizer**: annotate type of ABS *(PR [#2524](https://github.com/tobymao/sqlglot/pull/2524) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`c7302cf`](https://github.com/tobymao/sqlglot/commit/c7302cf54f71002878dce0f57c0c4a3b5aeafb41) - struct conversion for non correlated queries *(commit by [@tobymao](https://github.com/tobymao))*
- [`1aa727c`](https://github.com/tobymao/sqlglot/commit/1aa727c7578edc3f0be9f32fe5171ce9be185180) - subquery column lineage closes [#2510](https://github.com/tobymao/sqlglot/pull/2510) *(commit by [@tobymao](https://github.com/tobymao))*
- [`c6db124`](https://github.com/tobymao/sqlglot/commit/c6db1240b8481af0003a4bed4225f8e46578f182) - transpile division *(PR [#2513](https://github.com/tobymao/sqlglot/pull/2513) by [@barakalon](https://github.com/barakalon))*
- [`3787389`](https://github.com/tobymao/sqlglot/commit/3787389d4b369b910580931992d0133667d94969) - lineage with subquery and cte closes [#2515](https://github.com/tobymao/sqlglot/pull/2515) *(commit by [@tobymao](https://github.com/tobymao))*
- [`3469e75`](https://github.com/tobymao/sqlglot/commit/3469e75f6fc0c34d68c3a6f1a1b51d8cce3439ff) - typed div and safe div semantics *(PR [#2516](https://github.com/tobymao/sqlglot/pull/2516) by [@barakalon](https://github.com/barakalon))*
- [`52066ea`](https://github.com/tobymao/sqlglot/commit/52066ea899d1150cb2eaee71e606921f5f18bd09) - **tsql**: parse DROP CONSTRAINT into Drop instead of Command *(PR [#2521](https://github.com/tobymao/sqlglot/pull/2521) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2519](undefined) opened by [@HassanShafiq123](https://github.com/HassanShafiq123)*
- [`190f028`](https://github.com/tobymao/sqlglot/commit/190f028752bdcb8919ba79cef52dc96d11997975) - **oracle**: parse TO_CHAR using format_time_lambda *(PR [#2523](https://github.com/tobymao/sqlglot/pull/2523) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2517](undefined) opened by [@CBQu](https://github.com/CBQu)*
- [`e2e11ae`](https://github.com/tobymao/sqlglot/commit/e2e11ae2d8a8cfa5205b727cf748e01944c54d70) - **optimizer**: more support for date literals in simplify *(PR [#2525](https://github.com/tobymao/sqlglot/pull/2525) by [@barakalon](https://github.com/barakalon))*
### :wrench: Chores
- [`2065210`](https://github.com/tobymao/sqlglot/commit/206521070bd9f0156e1102f7bc93c452535a464b) - expressions type *(commit by [@tobymao](https://github.com/tobymao))*
- [`7ff5f25`](https://github.com/tobymao/sqlglot/commit/7ff5f254e755bfef02c694ca3920d10bc6e174cd) - use tuples instead of sets for inline collection instantiations *(PR [#2520](https://github.com/tobymao/sqlglot/pull/2520) by [@georgesittas](https://github.com/georgesittas))*
## [v19.0.3] - 2023-11-02
### :bug: Bug Fixes
- [`cdaf832`](https://github.com/tobymao/sqlglot/commit/cdaf832a392549f2910d97f1659c2dce0b0a6cf4) - **planner**: missing step of subquery *(PR [#2503](https://github.com/tobymao/sqlglot/pull/2503) by [@Thearas](https://github.com/Thearas))*
### :wrench: Chores
- [`c51d9ae`](https://github.com/tobymao/sqlglot/commit/c51d9ae7c45a66d32b4264712cc7d652964411fa) - fix type hint for normalize_identifiers *(PR [#2505](https://github.com/tobymao/sqlglot/pull/2505) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
## [v19.0.2] - 2023-11-01
### :sparkles: New Features
- [`3d60f0e`](https://github.com/tobymao/sqlglot/commit/3d60f0edd570ba51660110e3752d57a783b172ed) - **optimizer**: more date function coercion *(PR [#2493](https://github.com/tobymao/sqlglot/pull/2493) by [@barakalon](https://github.com/barakalon))*
- [`01c1abb`](https://github.com/tobymao/sqlglot/commit/01c1abb8fb1ff3d4e96c7aae203e3a137960b5a3) - **tsql**: add period constraint, system versioning property, generate as row *(PR [#2484](https://github.com/tobymao/sqlglot/pull/2484) by [@Rik-de-Kort](https://github.com/Rik-de-Kort))*
- [`b18235b`](https://github.com/tobymao/sqlglot/commit/b18235bb8a31a189cb634548db350282935a2991) - use parse_identifiers in qualify tables *(PR [#2502](https://github.com/tobymao/sqlglot/pull/2502) by [@tobymao](https://github.com/tobymao))*
### :bug: Bug Fixes
- [`471591c`](https://github.com/tobymao/sqlglot/commit/471591ce3e499d695541defdcfdc27cc07f01907) - **teradata**: support `TRYCAST` *(PR [#2496](https://github.com/tobymao/sqlglot/pull/2496) by [@hsheth2](https://github.com/hsheth2))*
- [`5cdbdf6`](https://github.com/tobymao/sqlglot/commit/5cdbdf6c102c37d91ee335cbe123a8aa3596e310) - **teradata**: support `**` for exponent *(PR [#2495](https://github.com/tobymao/sqlglot/pull/2495) by [@hsheth2](https://github.com/hsheth2))*
- [`70227fc`](https://github.com/tobymao/sqlglot/commit/70227fc8506d765ff7ac25606623a33ec0408284) - **postgres**: allow opclass types to be namespaced *(PR [#2499](https://github.com/tobymao/sqlglot/pull/2499) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2498](undefined) opened by [@Nitrino](https://github.com/Nitrino)*
- [`b434717`](https://github.com/tobymao/sqlglot/commit/b4347170891fac6c1fb29084032fe4dc01844711) - **tsql**: improve support for SYSTEM_VERSIONING *(PR [#2501](https://github.com/tobymao/sqlglot/pull/2501) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`8691e1a`](https://github.com/tobymao/sqlglot/commit/8691e1acaaaa2906da163e210593244f55b56093) - **oracle**: alter table add multiple columns closes [#2500](https://github.com/tobymao/sqlglot/pull/2500) *(commit by [@tobymao](https://github.com/tobymao))*
### :wrench: Chores
- [`808b0bb`](https://github.com/tobymao/sqlglot/commit/808b0bbc4781bd671f52169259434f7ad656e004) - skip branch if no exponent parsing *(commit by [@tobymao](https://github.com/tobymao))*
## [v19.0.1] - 2023-10-31
### :sparkles: New Features
- [`12596fd`](https://github.com/tobymao/sqlglot/commit/12596fdb83504b0e3e5f06132041771680c2560b) - support teradata as format no type closes [#2485](https://github.com/tobymao/sqlglot/pull/2485) *(PR [#2486](https://github.com/tobymao/sqlglot/pull/2486) by [@tobymao](https://github.com/tobymao))*
- [`f25b61c`](https://github.com/tobymao/sqlglot/commit/f25b61c084435f77e66dd980e9e4aec42f33b99d) - **redshift**: add support for DATE_DIFF *(PR [#2491](https://github.com/tobymao/sqlglot/pull/2491) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
### :bug: Bug Fixes
- [`83ecc5a`](https://github.com/tobymao/sqlglot/commit/83ecc5afb9be5ca3f7c6db3ca9f91e18aec93b88) - get rid of UNKNOWN type mapping in base Generator class *(PR [#2487](https://github.com/tobymao/sqlglot/pull/2487) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`22990ef`](https://github.com/tobymao/sqlglot/commit/22990efde00043ea1ede769aacc83f314ea920ae) - qualify catalog only if db is present *(PR [#2489](https://github.com/tobymao/sqlglot/pull/2489) by [@eakmanrq](https://github.com/eakmanrq))*
- [`8e20328`](https://github.com/tobymao/sqlglot/commit/8e203288adcbb45b23bfbb57416333ef6f91e370) - **schema**: use to_identifier as fallback when normalizing names *(PR [#2492](https://github.com/tobymao/sqlglot/pull/2492) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`f6c34b0`](https://github.com/tobymao/sqlglot/commit/f6c34b08d17b5c5c6aadeac362a4c85b4688b987) - **teradata**: add UPD and DEL abbreviations *(PR [#2494](https://github.com/tobymao/sqlglot/pull/2494) by [@hsheth2](https://github.com/hsheth2))*
## [v19.0.0] - 2023-10-30 ## [v19.0.0] - 2023-10-30
### :boom: BREAKING CHANGES ### :boom: BREAKING CHANGES
- due to [`7a6da28`](https://github.com/tobymao/sqlglot/commit/7a6da28907ef519b954bc813fe180c306fe5006b) - generator now always copies, making transforms much simpler *(PR [#2477](https://github.com/tobymao/sqlglot/pull/2477) by [@tobymao](https://github.com/tobymao))*: - due to [`7a6da28`](https://github.com/tobymao/sqlglot/commit/7a6da28907ef519b954bc813fe180c306fe5006b) - generator now always copies, making transforms much simpler *(PR [#2477](https://github.com/tobymao/sqlglot/pull/2477) by [@tobymao](https://github.com/tobymao))*:
@ -1844,3 +2187,24 @@ Changelog
[v18.16.1]: https://github.com/tobymao/sqlglot/compare/v18.16.0...v18.16.1 [v18.16.1]: https://github.com/tobymao/sqlglot/compare/v18.16.0...v18.16.1
[v18.17.0]: https://github.com/tobymao/sqlglot/compare/v18.16.1...v18.17.0 [v18.17.0]: https://github.com/tobymao/sqlglot/compare/v18.16.1...v18.17.0
[v19.0.0]: https://github.com/tobymao/sqlglot/compare/v18.17.0...v19.0.0 [v19.0.0]: https://github.com/tobymao/sqlglot/compare/v18.17.0...v19.0.0
[v19.0.1]: https://github.com/tobymao/sqlglot/compare/v19.0.0...v19.0.1
[v19.0.2]: https://github.com/tobymao/sqlglot/compare/v19.0.1...v19.0.2
[v19.0.3]: https://github.com/tobymao/sqlglot/compare/v19.0.2...v19.0.3
[v19.1.0]: https://github.com/tobymao/sqlglot/compare/v19.0.3...v19.1.0
[v19.1.1]: https://github.com/tobymao/sqlglot/compare/v19.1.0...v19.1.1
[v19.1.2]: https://github.com/tobymao/sqlglot/compare/v19.1.1...v19.1.2
[v19.1.3]: https://github.com/tobymao/sqlglot/compare/v19.1.2...v19.1.3
[v19.2.0]: https://github.com/tobymao/sqlglot/compare/v19.1.3...v19.2.0
[v19.3.0]: https://github.com/tobymao/sqlglot/compare/v19.2.0...v19.3.0
[v19.3.1]: https://github.com/tobymao/sqlglot/compare/v19.3.0...v19.3.1
[v19.4.0]: https://github.com/tobymao/sqlglot/compare/v19.3.1...v19.4.0
[v19.5.0]: https://github.com/tobymao/sqlglot/compare/v19.4.0...v19.5.0
[v19.5.1]: https://github.com/tobymao/sqlglot/compare/v19.5.0...v19.5.1
[v19.6.0]: https://github.com/tobymao/sqlglot/compare/v19.5.1...v19.6.0
[v19.7.0]: https://github.com/tobymao/sqlglot/compare/v19.6.0...v19.7.0
[v19.8.0]: https://github.com/tobymao/sqlglot/compare/v19.7.0...v19.8.0
[v19.8.1]: https://github.com/tobymao/sqlglot/compare/v19.8.0...v19.8.1
[v19.8.2]: https://github.com/tobymao/sqlglot/compare/v19.8.1...v19.8.2
[v19.8.3]: https://github.com/tobymao/sqlglot/compare/v19.8.2...v19.8.3
[v19.9.0]: https://github.com/tobymao/sqlglot/compare/v19.8.3...v19.9.0
[v20.0.0]: https://github.com/tobymao/sqlglot/compare/v19.9.0...v20.0.0

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

173
docs/sqlglot.html
View file

@ -697,68 +697,68 @@ make check # Full test suite &amp; linter checks
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">Expression</span> <span class="k">as</span> <span class="n">Expression</span><span class="p">,</span> </span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">Expression</span> <span class="k">as</span> <span class="n">Expression</span><span class="p">,</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">alias_</span> <span class="k">as</span> <span class="n">alias</span><span class="p">,</span> </span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">alias_</span> <span class="k">as</span> <span class="n">alias</span><span class="p">,</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="n">and_</span> <span class="k">as</span> <span class="n">and_</span><span class="p">,</span> </span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="n">and_</span> <span class="k">as</span> <span class="n">and_</span><span class="p">,</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">cast</span> <span class="k">as</span> <span class="n">cast</span><span class="p">,</span> </span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">case</span> <span class="k">as</span> <span class="n">case</span><span class="p">,</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">column</span> <span class="k">as</span> <span class="n">column</span><span class="p">,</span> </span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">cast</span> <span class="k">as</span> <span class="n">cast</span><span class="p">,</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="n">condition</span> <span class="k">as</span> <span class="n">condition</span><span class="p">,</span> </span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="n">column</span> <span class="k">as</span> <span class="n">column</span><span class="p">,</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">except_</span> <span class="k">as</span> <span class="n">except_</span><span class="p">,</span> </span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">condition</span> <span class="k">as</span> <span class="n">condition</span><span class="p">,</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">from_</span> <span class="k">as</span> <span class="n">from_</span><span class="p">,</span> </span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">except_</span> <span class="k">as</span> <span class="n">except_</span><span class="p">,</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="n">func</span> <span class="k">as</span> <span class="n">func</span><span class="p">,</span> </span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="n">from_</span> <span class="k">as</span> <span class="n">from_</span><span class="p">,</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="n">intersect</span> <span class="k">as</span> <span class="n">intersect</span><span class="p">,</span> </span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="n">func</span> <span class="k">as</span> <span class="n">func</span><span class="p">,</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">maybe_parse</span> <span class="k">as</span> <span class="n">maybe_parse</span><span class="p">,</span> </span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">intersect</span> <span class="k">as</span> <span class="n">intersect</span><span class="p">,</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="n">not_</span> <span class="k">as</span> <span class="n">not_</span><span class="p">,</span> </span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="n">maybe_parse</span> <span class="k">as</span> <span class="n">maybe_parse</span><span class="p">,</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="n">or_</span> <span class="k">as</span> <span class="n">or_</span><span class="p">,</span> </span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="n">not_</span> <span class="k">as</span> <span class="n">not_</span><span class="p">,</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="n">select</span> <span class="k">as</span> <span class="n">select</span><span class="p">,</span> </span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="n">or_</span> <span class="k">as</span> <span class="n">or_</span><span class="p">,</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="n">subquery</span> <span class="k">as</span> <span class="n">subquery</span><span class="p">,</span> </span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="n">select</span> <span class="k">as</span> <span class="n">select</span><span class="p">,</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="n">table_</span> <span class="k">as</span> <span class="n">table</span><span class="p">,</span> </span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="n">subquery</span> <span class="k">as</span> <span class="n">subquery</span><span class="p">,</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="n">to_column</span> <span class="k">as</span> <span class="n">to_column</span><span class="p">,</span> </span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="n">table_</span> <span class="k">as</span> <span class="n">table</span><span class="p">,</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="n">to_identifier</span> <span class="k">as</span> <span class="n">to_identifier</span><span class="p">,</span> </span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="n">to_column</span> <span class="k">as</span> <span class="n">to_column</span><span class="p">,</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="n">to_table</span> <span class="k">as</span> <span class="n">to_table</span><span class="p">,</span> </span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="n">to_identifier</span> <span class="k">as</span> <span class="n">to_identifier</span><span class="p">,</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="n">union</span> <span class="k">as</span> <span class="n">union</span><span class="p">,</span> </span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="n">to_table</span> <span class="k">as</span> <span class="n">to_table</span><span class="p">,</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="p">)</span> </span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="n">union</span> <span class="k">as</span> <span class="n">union</span><span class="p">,</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="kn">from</span> <span class="nn">sqlglot.generator</span> <span class="kn">import</span> <span class="n">Generator</span> <span class="k">as</span> <span class="n">Generator</span> </span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="p">)</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="kn">from</span> <span class="nn">sqlglot.parser</span> <span class="kn">import</span> <span class="n">Parser</span> <span class="k">as</span> <span class="n">Parser</span> </span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="kn">from</span> <span class="nn">sqlglot.generator</span> <span class="kn">import</span> <span class="n">Generator</span> <span class="k">as</span> <span class="n">Generator</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">MappingSchema</span> <span class="k">as</span> <span class="n">MappingSchema</span><span class="p">,</span> <span class="n">Schema</span> <span class="k">as</span> <span class="n">Schema</span> </span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="kn">from</span> <span class="nn">sqlglot.parser</span> <span class="kn">import</span> <span class="n">Parser</span> <span class="k">as</span> <span class="n">Parser</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="kn">from</span> <span class="nn">sqlglot.tokens</span> <span class="kn">import</span> <span class="n">Tokenizer</span> <span class="k">as</span> <span class="n">Tokenizer</span><span class="p">,</span> <span class="n">TokenType</span> <span class="k">as</span> <span class="n">TokenType</span> </span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">MappingSchema</span> <span class="k">as</span> <span class="n">MappingSchema</span><span class="p">,</span> <span class="n">Schema</span> <span class="k">as</span> <span class="n">Schema</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> </span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="kn">from</span> <span class="nn">sqlglot.tokens</span> <span class="kn">import</span> <span class="n">Tokenizer</span> <span class="k">as</span> <span class="n">Tokenizer</span><span class="p">,</span> <span class="n">TokenType</span> <span class="k">as</span> <span class="n">TokenType</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="k">if</span> <span class="n">t</span><span class="o">.</span><span class="n">TYPE_CHECKING</span><span class="p">:</span> </span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span> </span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a><span class="k">if</span> <span class="n">t</span><span class="o">.</span><span class="n">TYPE_CHECKING</span><span class="p">:</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">DialectType</span> <span class="k">as</span> <span class="n">DialectType</span> </span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> </span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">DialectType</span> <span class="k">as</span> <span class="n">DialectType</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">&quot;sqlglot&quot;</span><span class="p">)</span> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">&quot;sqlglot&quot;</span><span class="p">)</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="k">try</span><span class="p">:</span> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="kn">from</span> <span class="nn">sqlglot._version</span> <span class="kn">import</span> <span class="n">__version__</span><span class="p">,</span> <span class="n">__version_tuple__</span> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="k">try</span><span class="p">:</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a><span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="kn">from</span> <span class="nn">sqlglot._version</span> <span class="kn">import</span> <span class="n">__version__</span><span class="p">,</span> <span class="n">__version_tuple__</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a><span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="s2">&quot;Unable to set __version__, run `pip install -e .` or `python setup.py develop` first.&quot;</span> </span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="p">)</span> </span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="s2">&quot;Unable to set __version__, run `pip install -e .` or `python setup.py develop` first.&quot;</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> </span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> </span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a><span class="n">pretty</span> <span class="o">=</span> <span class="kc">False</span> </span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="sd">&quot;&quot;&quot;Whether to format generated SQL by default.&quot;&quot;&quot;</span> </span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="n">pretty</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> </span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a><span class="sd">&quot;&quot;&quot;Whether to format generated SQL by default.&quot;&quot;&quot;</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="n">schema</span> <span class="o">=</span> <span class="n">MappingSchema</span><span class="p">()</span> </span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="sd">&quot;&quot;&quot;The default schema used by SQLGlot (e.g. in the optimizer).&quot;&quot;&quot;</span> </span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="n">schema</span> <span class="o">=</span> <span class="n">MappingSchema</span><span class="p">()</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> </span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="sd">&quot;&quot;&quot;The default schema used by SQLGlot (e.g. in the optimizer).&quot;&quot;&quot;</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> </span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span> </span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span> </span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Expression</span><span class="p">]]:</span> </span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Expression</span><span class="p">]]:</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="sd"> Parses the given SQL string into a collection of syntax trees, one per parsed SQL statement.</span> </span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> </span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="sd"> Parses the given SQL string into a collection of syntax trees, one per parsed SQL statement.</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a><span class="sd"> Args:</span> </span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a><span class="sd"> sql: the SQL code string to parse.</span> </span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a><span class="sd"> Args:</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. &quot;spark&quot;, &quot;hive&quot;, &quot;presto&quot;, &quot;mysql&quot;).</span> </span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a><span class="sd"> sql: the SQL code string to parse.</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a><span class="sd"> dialect: the SQL dialect (alias for read).</span> </span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. &quot;spark&quot;, &quot;hive&quot;, &quot;presto&quot;, &quot;mysql&quot;).</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="sd"> **opts: other `sqlglot.parser.Parser` options.</span> </span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="sd"> dialect: the SQL dialect (alias for read).</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> </span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="sd"> **opts: other `sqlglot.parser.Parser` options.</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="sd"> Returns:</span> </span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="sd"> The resulting syntax tree collection.</span> </span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="sd"> Returns:</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="sd"> The resulting syntax tree collection.</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)()</span> </span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">return</span> <span class="n">dialect</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> </span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> </span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> </span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="nd">@t</span><span class="o">.</span><span class="n">overload</span> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="nd">@t</span><span class="o">.</span><span class="n">overload</span>
@ -792,7 +792,7 @@ make check # Full test suite &amp; linter checks
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a><span class="sd"> The syntax tree for the first parsed statement.</span> </span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a><span class="sd"> The syntax tree for the first parsed statement.</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> </span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)()</span> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">into</span><span class="p">:</span> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">into</span><span class="p">:</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">parse_into</span><span class="p">(</span><span class="n">into</span><span class="p">,</span> <span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">parse_into</span><span class="p">(</span><span class="n">into</span><span class="p">,</span> <span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span>
@ -832,10 +832,11 @@ make check # Full test suite &amp; linter checks
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a><span class="sd"> The list of transpiled SQL statements.</span> </span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a><span class="sd"> The list of transpiled SQL statements.</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span> </span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="p">[</span> </span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">write</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)()</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> <span class="k">if</span> <span class="n">expression</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span> </span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span> </span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">write</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> <span class="k">if</span> <span class="n">expression</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="p">]</span> </span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="p">]</span>
</span></pre></div> </span></pre></div>
@ -891,23 +892,22 @@ make check # Full test suite &amp; linter checks
</div> </div>
<a class="headerlink" href="#parse"></a> <a class="headerlink" href="#parse"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="parse-71"><a href="#parse-71"><span class="linenos">71</span></a><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span> <div class="pdoc-code codehilite"><pre><span></span><span id="parse-72"><a href="#parse-72"><span class="linenos">72</span></a><span class="k">def</span> <span class="nf">parse</span><span class="p">(</span>
</span><span id="parse-72"><a href="#parse-72"><span class="linenos">72</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span> </span><span id="parse-73"><a href="#parse-73"><span class="linenos">73</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span>
</span><span id="parse-73"><a href="#parse-73"><span class="linenos">73</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Expression</span><span class="p">]]:</span> </span><span id="parse-74"><a href="#parse-74"><span class="linenos">74</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Expression</span><span class="p">]]:</span>
</span><span id="parse-74"><a href="#parse-74"><span class="linenos">74</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="parse-75"><a href="#parse-75"><span class="linenos">75</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="parse-75"><a href="#parse-75"><span class="linenos">75</span></a><span class="sd"> Parses the given SQL string into a collection of syntax trees, one per parsed SQL statement.</span> </span><span id="parse-76"><a href="#parse-76"><span class="linenos">76</span></a><span class="sd"> Parses the given SQL string into a collection of syntax trees, one per parsed SQL statement.</span>
</span><span id="parse-76"><a href="#parse-76"><span class="linenos">76</span></a> </span><span id="parse-77"><a href="#parse-77"><span class="linenos">77</span></a>
</span><span id="parse-77"><a href="#parse-77"><span class="linenos">77</span></a><span class="sd"> Args:</span> </span><span id="parse-78"><a href="#parse-78"><span class="linenos">78</span></a><span class="sd"> Args:</span>
</span><span id="parse-78"><a href="#parse-78"><span class="linenos">78</span></a><span class="sd"> sql: the SQL code string to parse.</span> </span><span id="parse-79"><a href="#parse-79"><span class="linenos">79</span></a><span class="sd"> sql: the SQL code string to parse.</span>
</span><span id="parse-79"><a href="#parse-79"><span class="linenos">79</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. &quot;spark&quot;, &quot;hive&quot;, &quot;presto&quot;, &quot;mysql&quot;).</span> </span><span id="parse-80"><a href="#parse-80"><span class="linenos">80</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. &quot;spark&quot;, &quot;hive&quot;, &quot;presto&quot;, &quot;mysql&quot;).</span>
</span><span id="parse-80"><a href="#parse-80"><span class="linenos">80</span></a><span class="sd"> dialect: the SQL dialect (alias for read).</span> </span><span id="parse-81"><a href="#parse-81"><span class="linenos">81</span></a><span class="sd"> dialect: the SQL dialect (alias for read).</span>
</span><span id="parse-81"><a href="#parse-81"><span class="linenos">81</span></a><span class="sd"> **opts: other `sqlglot.parser.Parser` options.</span> </span><span id="parse-82"><a href="#parse-82"><span class="linenos">82</span></a><span class="sd"> **opts: other `sqlglot.parser.Parser` options.</span>
</span><span id="parse-82"><a href="#parse-82"><span class="linenos">82</span></a> </span><span id="parse-83"><a href="#parse-83"><span class="linenos">83</span></a>
</span><span id="parse-83"><a href="#parse-83"><span class="linenos">83</span></a><span class="sd"> Returns:</span> </span><span id="parse-84"><a href="#parse-84"><span class="linenos">84</span></a><span class="sd"> Returns:</span>
</span><span id="parse-84"><a href="#parse-84"><span class="linenos">84</span></a><span class="sd"> The resulting syntax tree collection.</span> </span><span id="parse-85"><a href="#parse-85"><span class="linenos">85</span></a><span class="sd"> The resulting syntax tree collection.</span>
</span><span id="parse-85"><a href="#parse-85"><span class="linenos">85</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="parse-86"><a href="#parse-86"><span class="linenos">86</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="parse-86"><a href="#parse-86"><span class="linenos">86</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)()</span> </span><span id="parse-87"><a href="#parse-87"><span class="linenos">87</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span>
</span><span id="parse-87"><a href="#parse-87"><span class="linenos">87</span></a> <span class="k">return</span> <span class="n">dialect</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -963,7 +963,7 @@ make check # Full test suite &amp; linter checks
</span><span id="parse_one-118"><a href="#parse_one-118"><span class="linenos">118</span></a><span class="sd"> The syntax tree for the first parsed statement.</span> </span><span id="parse_one-118"><a href="#parse_one-118"><span class="linenos">118</span></a><span class="sd"> The syntax tree for the first parsed statement.</span>
</span><span id="parse_one-119"><a href="#parse_one-119"><span class="linenos">119</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="parse_one-119"><a href="#parse_one-119"><span class="linenos">119</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="parse_one-120"><a href="#parse_one-120"><span class="linenos">120</span></a> </span><span id="parse_one-120"><a href="#parse_one-120"><span class="linenos">120</span></a>
</span><span id="parse_one-121"><a href="#parse_one-121"><span class="linenos">121</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)()</span> </span><span id="parse_one-121"><a href="#parse_one-121"><span class="linenos">121</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">read</span> <span class="ow">or</span> <span class="n">dialect</span><span class="p">)</span>
</span><span id="parse_one-122"><a href="#parse_one-122"><span class="linenos">122</span></a> </span><span id="parse_one-122"><a href="#parse_one-122"><span class="linenos">122</span></a>
</span><span id="parse_one-123"><a href="#parse_one-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="n">into</span><span class="p">:</span> </span><span id="parse_one-123"><a href="#parse_one-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="n">into</span><span class="p">:</span>
</span><span id="parse_one-124"><a href="#parse_one-124"><span class="linenos">124</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">parse_into</span><span class="p">(</span><span class="n">into</span><span class="p">,</span> <span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> </span><span id="parse_one-124"><a href="#parse_one-124"><span class="linenos">124</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">parse_into</span><span class="p">(</span><span class="n">into</span><span class="p">,</span> <span class="n">sql</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span>
@ -1036,10 +1036,11 @@ make check # Full test suite &amp; linter checks
</span><span id="transpile-158"><a href="#transpile-158"><span class="linenos">158</span></a><span class="sd"> The list of transpiled SQL statements.</span> </span><span id="transpile-158"><a href="#transpile-158"><span class="linenos">158</span></a><span class="sd"> The list of transpiled SQL statements.</span>
</span><span id="transpile-159"><a href="#transpile-159"><span class="linenos">159</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="transpile-159"><a href="#transpile-159"><span class="linenos">159</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="transpile-160"><a href="#transpile-160"><span class="linenos">160</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span> </span><span id="transpile-160"><a href="#transpile-160"><span class="linenos">160</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span>
</span><span id="transpile-161"><a href="#transpile-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="p">[</span> </span><span id="transpile-161"><a href="#transpile-161"><span class="linenos">161</span></a> <span class="n">write</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)</span>
</span><span id="transpile-162"><a href="#transpile-162"><span class="linenos">162</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)()</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> <span class="k">if</span> <span class="n">expression</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span> </span><span id="transpile-162"><a href="#transpile-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="transpile-163"><a href="#transpile-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span> </span><span id="transpile-163"><a href="#transpile-163"><span class="linenos">163</span></a> <span class="n">write</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> <span class="k">if</span> <span class="n">expression</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
</span><span id="transpile-164"><a href="#transpile-164"><span class="linenos">164</span></a> <span class="p">]</span> </span><span id="transpile-164"><a href="#transpile-164"><span class="linenos">164</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span>
</span><span id="transpile-165"><a href="#transpile-165"><span class="linenos">165</span></a> <span class="p">]</span>
</span></pre></div> </span></pre></div>

8
docs/sqlglot/_version.html
View file

@ -76,8 +76,8 @@
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> </span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;19.0.0&#39;</span> </span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;20.0.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">19</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> </span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">20</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -97,7 +97,7 @@
<section id="version"> <section id="version">
<div class="attr variable"> <div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> = <span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;19.0.0&#39;</span> <span class="default_value">&#39;20.0.0&#39;</span>
</div> </div>
@ -109,7 +109,7 @@
<section id="version_tuple"> <section id="version_tuple">
<div class="attr variable"> <div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> = <span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(19, 0, 0)</span> <span class="default_value">(20, 0, 0)</span>
</div> </div>

2261
docs/sqlglot/dataframe/sql.html
View file

File diff suppressed because it is too large Load diff

4352
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

3320
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

860
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

3934
docs/sqlglot/dialects/dialect.html
View file

File diff suppressed because it is too large Load diff

679
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

1105
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

2344
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

3285
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

4832
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

1649
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

2394
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

2589
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

1611
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

3764
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

725
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

1504
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

1470
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

410
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

382
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

1898
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

344
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

4566
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

168
docs/sqlglot/executor/env.html
View file

@ -63,6 +63,9 @@
<li> <li>
<a class="function" href="#interval">interval</a> <a class="function" href="#interval">interval</a>
</li> </li>
<li>
<a class="function" href="#arrayjoin">arrayjoin</a>
</li>
<li> <li>
<a class="variable" href="#ENV">ENV</a> <a class="variable" href="#ENV">ENV</a>
</li> </li>
@ -230,71 +233,83 @@
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">return</span> <span class="n">datetime</span><span class="o">.</span><span class="n">timedelta</span><span class="p">(</span><span class="o">**</span><span class="p">{</span><span class="n">unit</span><span class="p">:</span> <span class="nb">float</span><span class="p">(</span><span class="n">this</span><span class="p">)})</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">return</span> <span class="n">datetime</span><span class="o">.</span><span class="n">timedelta</span><span class="p">(</span><span class="o">**</span><span class="p">{</span><span class="n">unit</span><span class="p">:</span> <span class="nb">float</span><span class="p">(</span><span class="n">this</span><span class="p">)})</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a><span class="n">ENV</span> <span class="o">=</span> <span class="p">{</span> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a><span class="nd">@null_if_any</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="s2">&quot;expression&quot;</span><span class="p">)</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="s2">&quot;exp&quot;</span><span class="p">:</span> <span class="n">exp</span><span class="p">,</span> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a><span class="k">def</span> <span class="nf">arrayjoin</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">expression</span><span class="p">,</span> <span class="n">null</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="c1"># aggs</span> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="p">(</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">null</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">this</span><span class="p">)</span> <span class="k">if</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="s2">&quot;ARRAYAGG&quot;</span><span class="p">:</span> <span class="nb">list</span><span class="p">,</span> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="s2">&quot;AVG&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="n">statistics</span><span class="o">.</span><span class="n">fmean</span> <span class="k">if</span> <span class="n">PYTHON_VERSION</span> <span class="o">&gt;=</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">)</span> <span class="k">else</span> <span class="n">statistics</span><span class="o">.</span><span class="n">mean</span><span class="p">),</span> <span class="c1"># type: ignore</span> </span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="s2">&quot;COUNT&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="k">lambda</span> <span class="n">acc</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">acc</span><span class="p">),</span> <span class="kc">False</span><span class="p">),</span> </span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a><span class="n">ENV</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="s2">&quot;MAX&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="nb">max</span><span class="p">),</span> </span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="s2">&quot;exp&quot;</span><span class="p">:</span> <span class="n">exp</span><span class="p">,</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="s2">&quot;MIN&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="nb">min</span><span class="p">),</span> </span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="c1"># aggs</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="s2">&quot;SUM&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="nb">sum</span><span class="p">),</span> </span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="s2">&quot;ARRAYAGG&quot;</span><span class="p">:</span> <span class="nb">list</span><span class="p">,</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="c1"># scalar functions</span> </span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="s2">&quot;ARRAYUNIQUEAGG&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="k">lambda</span> <span class="n">acc</span><span class="p">:</span> <span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">acc</span><span class="p">))),</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="s2">&quot;ABS&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">:</span> <span class="nb">abs</span><span class="p">(</span><span class="n">this</span><span class="p">)),</span> </span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="s2">&quot;AVG&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="n">statistics</span><span class="o">.</span><span class="n">fmean</span> <span class="k">if</span> <span class="n">PYTHON_VERSION</span> <span class="o">&gt;=</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">)</span> <span class="k">else</span> <span class="n">statistics</span><span class="o">.</span><span class="n">mean</span><span class="p">),</span> <span class="c1"># type: ignore</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="s2">&quot;ADD&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">+</span> <span class="n">this</span><span class="p">),</span> </span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="s2">&quot;COUNT&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="k">lambda</span> <span class="n">acc</span><span class="p">:</span> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">acc</span><span class="p">),</span> <span class="kc">False</span><span class="p">),</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="s2">&quot;ARRAYANY&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arr</span><span class="p">,</span> <span class="n">func</span><span class="p">:</span> <span class="nb">any</span><span class="p">(</span><span class="n">func</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">arr</span><span class="p">)),</span> </span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="s2">&quot;MAX&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="nb">max</span><span class="p">),</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="s2">&quot;BETWEEN&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">low</span><span class="p">,</span> <span class="n">high</span><span class="p">:</span> <span class="n">low</span> <span class="o">&lt;=</span> <span class="n">this</span> <span class="ow">and</span> <span class="n">this</span> <span class="o">&lt;=</span> <span class="n">high</span><span class="p">),</span> </span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="s2">&quot;MIN&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="nb">min</span><span class="p">),</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="s2">&quot;BITWISEAND&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&amp;</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="s2">&quot;SUM&quot;</span><span class="p">:</span> <span class="n">filter_nulls</span><span class="p">(</span><span class="nb">sum</span><span class="p">),</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="s2">&quot;BITWISELEFTSHIFT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&lt;&lt;</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="c1"># scalar functions</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="s2">&quot;BITWISEOR&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">|</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="s2">&quot;ABS&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">:</span> <span class="nb">abs</span><span class="p">(</span><span class="n">this</span><span class="p">)),</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="s2">&quot;BITWISERIGHTSHIFT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&gt;&gt;</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="s2">&quot;ADD&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">+</span> <span class="n">this</span><span class="p">),</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="s2">&quot;BITWISEXOR&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">^</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="s2">&quot;ARRAYANY&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arr</span><span class="p">,</span> <span class="n">func</span><span class="p">:</span> <span class="nb">any</span><span class="p">(</span><span class="n">func</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">arr</span><span class="p">)),</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="s2">&quot;CAST&quot;</span><span class="p">:</span> <span class="n">cast</span><span class="p">,</span> </span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="s2">&quot;ARRAYJOIN&quot;</span><span class="p">:</span> <span class="n">arrayjoin</span><span class="p">,</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="s2">&quot;COALESCE&quot;</span><span class="p">:</span> <span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="nb">next</span><span class="p">((</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">args</span> <span class="k">if</span> <span class="n">a</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">),</span> <span class="kc">None</span><span class="p">),</span> </span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="s2">&quot;BETWEEN&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">low</span><span class="p">,</span> <span class="n">high</span><span class="p">:</span> <span class="n">low</span> <span class="o">&lt;=</span> <span class="n">this</span> <span class="ow">and</span> <span class="n">this</span> <span class="o">&lt;=</span> <span class="n">high</span><span class="p">),</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="s2">&quot;CONCAT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">args</span><span class="p">)),</span> </span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="s2">&quot;BITWISEAND&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&amp;</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="s2">&quot;SAFECONCAT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">args</span><span class="p">)),</span> </span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="s2">&quot;BITWISELEFTSHIFT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&lt;&lt;</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="s2">&quot;CONCATWS&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">this</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">args</span><span class="p">)),</span> </span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="s2">&quot;BITWISEOR&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">|</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="s2">&quot;DATEDIFF&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">expression</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="p">(</span><span class="n">this</span> <span class="o">-</span> <span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">days</span><span class="p">),</span> </span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="s2">&quot;BITWISERIGHTSHIFT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&gt;&gt;</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="s2">&quot;DATESTRTODATE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">fromisoformat</span><span class="p">(</span><span class="n">arg</span><span class="p">)),</span> </span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="s2">&quot;BITWISEXOR&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">^</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="s2">&quot;DIV&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">/</span> <span class="n">this</span><span class="p">),</span> </span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="s2">&quot;CAST&quot;</span><span class="p">:</span> <span class="n">cast</span><span class="p">,</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="s2">&quot;DOT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span><span class="p">[</span><span class="n">this</span><span class="p">]),</span> </span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="s2">&quot;COALESCE&quot;</span><span class="p">:</span> <span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="nb">next</span><span class="p">((</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">args</span> <span class="k">if</span> <span class="n">a</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">),</span> <span class="kc">None</span><span class="p">),</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="s2">&quot;EQ&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">==</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="s2">&quot;CONCAT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">args</span><span class="p">)),</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="s2">&quot;EXTRACT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">)),</span> </span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="s2">&quot;SAFECONCAT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">args</span><span class="p">)),</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="s2">&quot;GT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&gt;</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="s2">&quot;CONCATWS&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">this</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">args</span><span class="p">)),</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="s2">&quot;GTE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&gt;=</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="s2">&quot;DATEDIFF&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">expression</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="p">(</span><span class="n">this</span> <span class="o">-</span> <span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">days</span><span class="p">),</span>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="s2">&quot;IF&quot;</span><span class="p">:</span> <span class="k">lambda</span> <span class="n">predicate</span><span class="p">,</span> <span class="n">true</span><span class="p">,</span> <span class="n">false</span><span class="p">:</span> <span class="n">true</span> <span class="k">if</span> <span class="n">predicate</span> <span class="k">else</span> <span class="n">false</span><span class="p">,</span> </span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="s2">&quot;DATESTRTODATE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">fromisoformat</span><span class="p">(</span><span class="n">arg</span><span class="p">)),</span>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="s2">&quot;INTDIV&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">//</span> <span class="n">this</span><span class="p">),</span> </span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="s2">&quot;DIV&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">/</span> <span class="n">this</span><span class="p">),</span>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="s2">&quot;INTERVAL&quot;</span><span class="p">:</span> <span class="n">interval</span><span class="p">,</span> </span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="s2">&quot;DOT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span><span class="p">[</span><span class="n">this</span><span class="p">]),</span>
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="s2">&quot;LEFT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span><span class="p">[:</span><span class="n">e</span><span class="p">]),</span> </span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="s2">&quot;EQ&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">==</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="s2">&quot;LIKE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span> </span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="s2">&quot;EXTRACT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">)),</span>
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="nb">bool</span><span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">&quot;_&quot;</span><span class="p">,</span> <span class="s2">&quot;.&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">&quot;%&quot;</span><span class="p">,</span> <span class="s2">&quot;.*&quot;</span><span class="p">),</span> <span class="n">this</span><span class="p">))</span> </span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="s2">&quot;GT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&gt;</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="p">),</span> </span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="s2">&quot;GTE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&gt;=</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="s2">&quot;LOWER&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">lower</span><span class="p">()),</span> </span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="s2">&quot;IF&quot;</span><span class="p">:</span> <span class="k">lambda</span> <span class="n">predicate</span><span class="p">,</span> <span class="n">true</span><span class="p">,</span> <span class="n">false</span><span class="p">:</span> <span class="n">true</span> <span class="k">if</span> <span class="n">predicate</span> <span class="k">else</span> <span class="n">false</span><span class="p">,</span>
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="s2">&quot;LT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&lt;</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="s2">&quot;INTDIV&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">//</span> <span class="n">this</span><span class="p">),</span>
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="s2">&quot;LTE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&lt;=</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="s2">&quot;INTERVAL&quot;</span><span class="p">:</span> <span class="n">interval</span><span class="p">,</span>
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="s2">&quot;MAP&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="nb">dict</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">))),</span> <span class="c1"># type: ignore</span> </span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="s2">&quot;LEFT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span><span class="p">[:</span><span class="n">e</span><span class="p">]),</span>
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="s2">&quot;MOD&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">%</span> <span class="n">this</span><span class="p">),</span> </span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="s2">&quot;LIKE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span>
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="s2">&quot;MUL&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">*</span> <span class="n">this</span><span class="p">),</span> </span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="nb">bool</span><span class="p">(</span><span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">&quot;_&quot;</span><span class="p">,</span> <span class="s2">&quot;.&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">&quot;%&quot;</span><span class="p">,</span> <span class="s2">&quot;.*&quot;</span><span class="p">),</span> <span class="n">this</span><span class="p">))</span>
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="s2">&quot;NEQ&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">!=</span> <span class="n">e</span><span class="p">),</span> </span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="p">),</span>
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="s2">&quot;ORD&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="nb">ord</span><span class="p">),</span> </span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="s2">&quot;LOWER&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">lower</span><span class="p">()),</span>
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="s2">&quot;ORDERED&quot;</span><span class="p">:</span> <span class="n">ordered</span><span class="p">,</span> </span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="s2">&quot;LT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&lt;</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="s2">&quot;POW&quot;</span><span class="p">:</span> <span class="nb">pow</span><span class="p">,</span> </span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="s2">&quot;LTE&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">&lt;=</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span><span class="p">[</span><span class="o">-</span><span class="n">e</span><span class="p">:]),</span> </span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="s2">&quot;MAP&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="nb">dict</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">))),</span> <span class="c1"># type: ignore</span>
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="s2">&quot;STRPOSITION&quot;</span><span class="p">:</span> <span class="n">str_position</span><span class="p">,</span> </span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="s2">&quot;MOD&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">%</span> <span class="n">this</span><span class="p">),</span>
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="s2">&quot;SUB&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">-</span> <span class="n">this</span><span class="p">),</span> </span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="s2">&quot;MUL&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">*</span> <span class="n">this</span><span class="p">),</span>
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="s2">&quot;SUBSTRING&quot;</span><span class="p">:</span> <span class="n">substring</span><span class="p">,</span> </span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="s2">&quot;NEQ&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span> <span class="o">!=</span> <span class="n">e</span><span class="p">),</span>
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="s2">&quot;TIMESTRTOTIME&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">fromisoformat</span><span class="p">(</span><span class="n">arg</span><span class="p">)),</span> </span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="s2">&quot;ORD&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="nb">ord</span><span class="p">),</span>
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="s2">&quot;UPPER&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">upper</span><span class="p">()),</span> </span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="s2">&quot;ORDERED&quot;</span><span class="p">:</span> <span class="n">ordered</span><span class="p">,</span>
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="s2">&quot;YEAR&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">year</span><span class="p">),</span> </span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="s2">&quot;POW&quot;</span><span class="p">:</span> <span class="nb">pow</span><span class="p">,</span>
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="s2">&quot;MONTH&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">month</span><span class="p">),</span> </span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="n">this</span><span class="p">[</span><span class="o">-</span><span class="n">e</span><span class="p">:]),</span>
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="s2">&quot;DAY&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">day</span><span class="p">),</span> </span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="s2">&quot;STRPOSITION&quot;</span><span class="p">:</span> <span class="n">str_position</span><span class="p">,</span>
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="s2">&quot;CURRENTDATETIME&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">,</span> </span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="s2">&quot;SUB&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="n">this</span><span class="p">:</span> <span class="n">e</span> <span class="o">-</span> <span class="n">this</span><span class="p">),</span>
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="s2">&quot;CURRENTTIMESTAMP&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">,</span> </span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="s2">&quot;SUBSTRING&quot;</span><span class="p">:</span> <span class="n">substring</span><span class="p">,</span>
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="s2">&quot;CURRENTTIME&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">,</span> </span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="s2">&quot;TIMESTRTOTIME&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">fromisoformat</span><span class="p">(</span><span class="n">arg</span><span class="p">)),</span>
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="s2">&quot;CURRENTDATE&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">today</span><span class="p">,</span> </span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="s2">&quot;UPPER&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">upper</span><span class="p">()),</span>
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="s2">&quot;STRFTIME&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">fmt</span><span class="p">,</span> <span class="n">arg</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">fromisoformat</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">fmt</span><span class="p">)),</span> </span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="s2">&quot;YEAR&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">year</span><span class="p">),</span>
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="s2">&quot;TRIM&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="o">=</span><span class="kc">None</span><span class="p">:</span> <span class="n">this</span><span class="o">.</span><span class="n">strip</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span> </span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="s2">&quot;MONTH&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">month</span><span class="p">),</span>
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a><span class="p">}</span> </span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="s2">&quot;DAY&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">arg</span><span class="p">:</span> <span class="n">arg</span><span class="o">.</span><span class="n">day</span><span class="p">),</span>
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="s2">&quot;CURRENTDATETIME&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">,</span>
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="s2">&quot;CURRENTTIMESTAMP&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">,</span>
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="s2">&quot;CURRENTTIME&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">,</span>
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="s2">&quot;CURRENTDATE&quot;</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">today</span><span class="p">,</span>
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="s2">&quot;STRFTIME&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">fmt</span><span class="p">,</span> <span class="n">arg</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">datetime</span><span class="o">.</span><span class="n">fromisoformat</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">fmt</span><span class="p">)),</span>
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="s2">&quot;TRIM&quot;</span><span class="p">:</span> <span class="n">null_if_any</span><span class="p">(</span><span class="k">lambda</span> <span class="n">this</span><span class="p">,</span> <span class="n">e</span><span class="o">=</span><span class="kc">None</span><span class="p">:</span> <span class="n">this</span><span class="o">.</span><span class="n">strip</span><span class="p">(</span><span class="n">e</span><span class="p">)),</span>
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="s2">&quot;STRUCT&quot;</span><span class="p">:</span> <span class="k">lambda</span> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="n">args</span><span class="p">[</span><span class="n">x</span><span class="p">]:</span> <span class="n">args</span><span class="p">[</span><span class="n">x</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span>
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">args</span><span class="p">),</span> <span class="mi">2</span><span class="p">)</span>
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">args</span><span class="p">[</span><span class="n">x</span> <span class="o">+</span> <span class="mi">1</span><span class="p">]</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="n">args</span><span class="p">[</span><span class="n">x</span><span class="p">]</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="p">},</span>
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a><span class="p">}</span>
</span></pre></div> </span></pre></div>
@ -591,12 +606,33 @@ def foo(a, b): ...
</section>
<section id="arrayjoin">
<input id="arrayjoin-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function">
<div class="decorator">@null_if_any(&#39;this&#39;, &#39;expression&#39;)</div>
<span class="def">def</span>
<span class="name">arrayjoin</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">this</span>, </span><span class="param"><span class="n">expression</span>, </span><span class="param"><span class="n">null</span><span class="o">=</span><span class="kc">None</span></span><span class="return-annotation">):</span></span>
<label class="view-source-button" for="arrayjoin-view-source"><span>View Source</span></label>
</div>
<a class="headerlink" href="#arrayjoin"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="arrayjoin-143"><a href="#arrayjoin-143"><span class="linenos">143</span></a><span class="nd">@null_if_any</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="s2">&quot;expression&quot;</span><span class="p">)</span>
</span><span id="arrayjoin-144"><a href="#arrayjoin-144"><span class="linenos">144</span></a><span class="k">def</span> <span class="nf">arrayjoin</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">expression</span><span class="p">,</span> <span class="n">null</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="arrayjoin-145"><a href="#arrayjoin-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">x</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="p">(</span><span class="n">x</span> <span class="k">if</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">null</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">this</span><span class="p">)</span> <span class="k">if</span> <span class="n">x</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">)</span>
</span></pre></div>
</section> </section>
<section id="ENV"> <section id="ENV">
<div class="attr variable"> <div class="attr variable">
<span class="name">ENV</span> = <span class="name">ENV</span> =
<input id="ENV-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="ENV-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ENV-view-value"></label><span class="default_value">{&#39;exp&#39;: &lt;module &#39;<a href="../expressions.html">sqlglot.expressions</a>&#39; from &#39;/home/runner/work/sqlglot/sqlglot/sqlglot/expressions.py&#39;&gt;, &#39;ARRAYAGG&#39;: &lt;class &#39;list&#39;&gt;, &#39;AVG&#39;: &lt;function fmean&gt;, &#39;COUNT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MAX&#39;: &lt;function max&gt;, &#39;MIN&#39;: &lt;function min&gt;, &#39;SUM&#39;: &lt;function sum&gt;, &#39;ABS&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ADD&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ARRAYANY&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BETWEEN&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISEAND&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISELEFTSHIFT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISEOR&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISERIGHTSHIFT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISEXOR&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CAST&#39;: &lt;function cast&gt;, &#39;COALESCE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CONCAT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;SAFECONCAT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CONCATWS&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DATEDIFF&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DATESTRTODATE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DIV&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DOT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;EQ&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;EXTRACT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;GT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;GTE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;IF&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;INTDIV&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;INTERVAL&#39;: &lt;function interval&gt;, &#39;LEFT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LIKE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LOWER&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LTE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MAP&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MOD&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MUL&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;NEQ&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ORD&#39;: &lt;function ord&gt;, &#39;ORDERED&#39;: &lt;function ordered&gt;, &#39;POW&#39;: &lt;built-in function pow&gt;, &#39;RIGHT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;STRPOSITION&#39;: &lt;function str_position&gt;, &#39;SUB&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;SUBSTRING&#39;: &lt;function substring&gt;, &#39;TIMESTRTOTIME&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;UPPER&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;YEAR&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MONTH&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DAY&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CURRENTDATETIME&#39;: &lt;built-in method now of type object&gt;, &#39;CURRENTTIMESTAMP&#39;: &lt;built-in method now of type object&gt;, &#39;CURRENTTIME&#39;: &lt;built-in method now of type object&gt;, &#39;CURRENTDATE&#39;: &lt;built-in method today of type object&gt;, &#39;STRFTIME&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;TRIM&#39;: &lt;function &lt;lambda&gt;&gt;}</span> <label class="view-value-button pdoc-button" for="ENV-view-value"></label><span class="default_value">{&#39;exp&#39;: &lt;module &#39;<a href="../expressions.html">sqlglot.expressions</a>&#39; from &#39;/home/runner/work/sqlglot/sqlglot/sqlglot/expressions.py&#39;&gt;, &#39;ARRAYAGG&#39;: &lt;class &#39;list&#39;&gt;, &#39;ARRAYUNIQUEAGG&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;AVG&#39;: &lt;function fmean&gt;, &#39;COUNT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MAX&#39;: &lt;function max&gt;, &#39;MIN&#39;: &lt;function min&gt;, &#39;SUM&#39;: &lt;function sum&gt;, &#39;ABS&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ADD&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ARRAYANY&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ARRAYJOIN&#39;: &lt;function arrayjoin&gt;, &#39;BETWEEN&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISEAND&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISELEFTSHIFT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISEOR&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISERIGHTSHIFT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;BITWISEXOR&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CAST&#39;: &lt;function cast&gt;, &#39;COALESCE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CONCAT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;SAFECONCAT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CONCATWS&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DATEDIFF&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DATESTRTODATE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DIV&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DOT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;EQ&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;EXTRACT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;GT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;GTE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;IF&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;INTDIV&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;INTERVAL&#39;: &lt;function interval&gt;, &#39;LEFT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LIKE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LOWER&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;LTE&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MAP&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MOD&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MUL&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;NEQ&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;ORD&#39;: &lt;function ord&gt;, &#39;ORDERED&#39;: &lt;function ordered&gt;, &#39;POW&#39;: &lt;built-in function pow&gt;, &#39;RIGHT&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;STRPOSITION&#39;: &lt;function str_position&gt;, &#39;SUB&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;SUBSTRING&#39;: &lt;function substring&gt;, &#39;TIMESTRTOTIME&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;UPPER&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;YEAR&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;MONTH&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;DAY&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;CURRENTDATETIME&#39;: &lt;built-in method now of type object&gt;, &#39;CURRENTTIMESTAMP&#39;: &lt;built-in method now of type object&gt;, &#39;CURRENTTIME&#39;: &lt;built-in method now of type object&gt;, &#39;CURRENTDATE&#39;: &lt;built-in method today of type object&gt;, &#39;STRFTIME&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;TRIM&#39;: &lt;function &lt;lambda&gt;&gt;, &#39;STRUCT&#39;: &lt;function &lt;lambda&gt;&gt;}</span>
</div> </div>

522
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

48
docs/sqlglot/executor/table.html
View file

@ -265,29 +265,31 @@
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">depth</span> <span class="o">=</span> <span class="n">dict_depth</span><span class="p">(</span><span class="n">d</span><span class="p">)</span> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">depth</span> <span class="o">=</span> <span class="n">dict_depth</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="p">{</span> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="p">{</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span> <span class="n">_ensure_tables</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">_ensure_tables</span><span class="p">(</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> </span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">v</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="p">}</span> </span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="p">)</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> </span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span> </span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">}</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">normalize_name</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> </span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">Table</span><span class="p">):</span> </span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">normalize_name</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span> </span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">Table</span><span class="p">):</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">table</span> <span class="o">=</span> <span class="p">[</span> </span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="p">{</span> </span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">):</span> <span class="n">value</span> </span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">table</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">row</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> </span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="p">{</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="p">}</span> </span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">value</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">row</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">]</span> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">}</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">column_name</span> <span class="k">for</span> <span class="n">column_name</span> <span class="ow">in</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">table</span> <span class="k">else</span> <span class="p">()</span> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">rows</span> <span class="o">=</span> <span class="p">[</span><span class="nb">tuple</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">column_names</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="p">]</span> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="p">]</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">column_names</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="n">rows</span><span class="p">)</span> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">column_name</span> <span class="k">for</span> <span class="n">column_name</span> <span class="ow">in</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">table</span> <span class="k">else</span> <span class="p">()</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">rows</span> <span class="o">=</span> <span class="p">[</span><span class="nb">tuple</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">column_names</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="p">]</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">result</span> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">column_names</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="n">rows</span><span class="p">)</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">result</span>
</span></pre></div> </span></pre></div>

29412
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

16857
docs/sqlglot/generator.html
View file

File diff suppressed because one or more lines are too long

1744
docs/sqlglot/helper.html
View file

File diff suppressed because it is too large Load diff

1275
docs/sqlglot/lineage.html
View file

File diff suppressed because it is too large Load diff

2098
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

441
docs/sqlglot/optimizer/canonicalize.html
View file

@ -39,6 +39,9 @@
<li> <li>
<a class="function" href="#replace_date_funcs">replace_date_funcs</a> <a class="function" href="#replace_date_funcs">replace_date_funcs</a>
</li> </li>
<li>
<a class="variable" href="#COERCIBLE_DATE_OPS">COERCIBLE_DATE_OPS</a>
</li>
<li> <li>
<a class="function" href="#coerce_type">coerce_type</a> <a class="function" href="#coerce_type">coerce_type</a>
</li> </li>
@ -46,7 +49,7 @@
<a class="function" href="#remove_redundant_casts">remove_redundant_casts</a> <a class="function" href="#remove_redundant_casts">remove_redundant_casts</a>
</li> </li>
<li> <li>
<a class="function" href="#ensure_bool_predicates">ensure_bool_predicates</a> <a class="function" href="#ensure_bools">ensure_bools</a>
</li> </li>
<li> <li>
<a class="function" href="#remove_ascending_order">remove_ascending_order</a> <a class="function" href="#remove_ascending_order">remove_ascending_order</a>
@ -77,113 +80,174 @@
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">annotations</span> <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">annotations</span>
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a> </span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">itertools</span> </span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">itertools</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a> </span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span> </span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a> </span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a> </span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">is_date_unit</span><span class="p">,</span> <span class="n">is_iso_date</span><span class="p">,</span> <span class="n">is_iso_datetime</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="k">def</span> <span class="nf">canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Converts a sql expression into a standard form.</span> </span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a> </span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="k">def</span> <span class="nf">canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="sd"> This method relies on annotate_types because many of the</span> </span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Converts a sql expression into a standard form.</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="sd"> conversions rely on type inference.</span> </span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> </span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="sd"> This method relies on annotate_types because many of the</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="sd"> Args:</span> </span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="sd"> conversions rely on type inference.</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="sd"> expression: The expression to canonicalize.</span> </span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="sd"> Args:</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">canonicalize</span><span class="p">)</span> </span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="sd"> expression: The expression to canonicalize.</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> </span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">canonicalize</span><span class="p">)</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bool_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> </span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bools</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">_replace_int_predicate</span><span class="p">)</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> </span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> </span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="k">def</span> <span class="nf">add_text_to_concat</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">)</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span> </span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Concat</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">])</span> </span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="k">def</span> <span class="nf">add_text_to_concat</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">)</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> </span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Concat</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">])</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> </span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="k">def</span> <span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span> </span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span> </span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="k">def</span> <span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span> </span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span> </span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> </span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> </span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span> </span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span> </span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span> </span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span> </span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span> </span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span> </span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="p">):</span> </span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span> </span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span> </span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span> </span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> </span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="p">):</span> </span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span> </span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> </span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> </span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="p">):</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="k">def</span> <span class="nf">ensure_bool_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span> </span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span> </span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span> </span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> </span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">))</span> <span class="ow">or</span> <span class="p">(</span> </span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span> </span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> </span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span><span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> </span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="p">):</span> </span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> </span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> </span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> </span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> </span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="p">):</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span> </span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span> </span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> </span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> </span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span> </span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span> </span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span> </span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> </span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span> </span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">INTERVAL</span><span class="p">)</span> </span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="p">):</span> </span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span> </span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">):</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> </span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> </span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span> </span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span> </span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> </span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> </span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span> </span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span> </span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span> </span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span> </span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span> </span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="o">.</span><span class="n">number</span><span class="p">(</span><span class="mi">0</span><span class="p">)))</span> </span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="ow">not</span> <span class="ow">in</span> <span class="p">(</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">,</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">INTERVAL</span><span class="p">,</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="p">)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">):</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a><span class="c1"># presto has a boolean type whereas tsql doesn&#39;t (people use bits)</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
</span></pre></div> </span></pre></div>
@ -199,25 +263,25 @@
</div> </div>
<a class="headerlink" href="#canonicalize"></a> <a class="headerlink" href="#canonicalize"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="canonicalize-9"><a href="#canonicalize-9"><span class="linenos"> 9</span></a><span class="k">def</span> <span class="nf">canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="canonicalize-11"><a href="#canonicalize-11"><span class="linenos">11</span></a><span class="k">def</span> <span class="nf">canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="canonicalize-10"><a href="#canonicalize-10"><span class="linenos">10</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Converts a sql expression into a standard form.</span> </span><span id="canonicalize-12"><a href="#canonicalize-12"><span class="linenos">12</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Converts a sql expression into a standard form.</span>
</span><span id="canonicalize-11"><a href="#canonicalize-11"><span class="linenos">11</span></a> </span><span id="canonicalize-13"><a href="#canonicalize-13"><span class="linenos">13</span></a>
</span><span id="canonicalize-12"><a href="#canonicalize-12"><span class="linenos">12</span></a><span class="sd"> This method relies on annotate_types because many of the</span> </span><span id="canonicalize-14"><a href="#canonicalize-14"><span class="linenos">14</span></a><span class="sd"> This method relies on annotate_types because many of the</span>
</span><span id="canonicalize-13"><a href="#canonicalize-13"><span class="linenos">13</span></a><span class="sd"> conversions rely on type inference.</span> </span><span id="canonicalize-15"><a href="#canonicalize-15"><span class="linenos">15</span></a><span class="sd"> conversions rely on type inference.</span>
</span><span id="canonicalize-14"><a href="#canonicalize-14"><span class="linenos">14</span></a> </span><span id="canonicalize-16"><a href="#canonicalize-16"><span class="linenos">16</span></a>
</span><span id="canonicalize-15"><a href="#canonicalize-15"><span class="linenos">15</span></a><span class="sd"> Args:</span> </span><span id="canonicalize-17"><a href="#canonicalize-17"><span class="linenos">17</span></a><span class="sd"> Args:</span>
</span><span id="canonicalize-16"><a href="#canonicalize-16"><span class="linenos">16</span></a><span class="sd"> expression: The expression to canonicalize.</span> </span><span id="canonicalize-18"><a href="#canonicalize-18"><span class="linenos">18</span></a><span class="sd"> expression: The expression to canonicalize.</span>
</span><span id="canonicalize-17"><a href="#canonicalize-17"><span class="linenos">17</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="canonicalize-19"><a href="#canonicalize-19"><span class="linenos">19</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="canonicalize-18"><a href="#canonicalize-18"><span class="linenos">18</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">canonicalize</span><span class="p">)</span> </span><span id="canonicalize-20"><a href="#canonicalize-20"><span class="linenos">20</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">canonicalize</span><span class="p">)</span>
</span><span id="canonicalize-19"><a href="#canonicalize-19"><span class="linenos">19</span></a> </span><span id="canonicalize-21"><a href="#canonicalize-21"><span class="linenos">21</span></a>
</span><span id="canonicalize-20"><a href="#canonicalize-20"><span class="linenos">20</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-22"><a href="#canonicalize-22"><span class="linenos">22</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-21"><a href="#canonicalize-21"><span class="linenos">21</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-23"><a href="#canonicalize-23"><span class="linenos">23</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-22"><a href="#canonicalize-22"><span class="linenos">22</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-24"><a href="#canonicalize-24"><span class="linenos">24</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-23"><a href="#canonicalize-23"><span class="linenos">23</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-25"><a href="#canonicalize-25"><span class="linenos">25</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-24"><a href="#canonicalize-24"><span class="linenos">24</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bool_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-26"><a href="#canonicalize-26"><span class="linenos">26</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bools</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">_replace_int_predicate</span><span class="p">)</span>
</span><span id="canonicalize-25"><a href="#canonicalize-25"><span class="linenos">25</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-27"><a href="#canonicalize-27"><span class="linenos">27</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-26"><a href="#canonicalize-26"><span class="linenos">26</span></a> </span><span id="canonicalize-28"><a href="#canonicalize-28"><span class="linenos">28</span></a>
</span><span id="canonicalize-27"><a href="#canonicalize-27"><span class="linenos">27</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="canonicalize-29"><a href="#canonicalize-29"><span class="linenos">29</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -246,10 +310,10 @@ conversions rely on type inference.</p>
</div> </div>
<a class="headerlink" href="#add_text_to_concat"></a> <a class="headerlink" href="#add_text_to_concat"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="add_text_to_concat-30"><a href="#add_text_to_concat-30"><span class="linenos">30</span></a><span class="k">def</span> <span class="nf">add_text_to_concat</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="add_text_to_concat-32"><a href="#add_text_to_concat-32"><span class="linenos">32</span></a><span class="k">def</span> <span class="nf">add_text_to_concat</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="add_text_to_concat-31"><a href="#add_text_to_concat-31"><span class="linenos">31</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">)</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span> </span><span id="add_text_to_concat-33"><a href="#add_text_to_concat-33"><span class="linenos">33</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">)</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="add_text_to_concat-32"><a href="#add_text_to_concat-32"><span class="linenos">32</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Concat</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">])</span> </span><span id="add_text_to_concat-34"><a href="#add_text_to_concat-34"><span class="linenos">34</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Concat</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">])</span>
</span><span id="add_text_to_concat-33"><a href="#add_text_to_concat-33"><span class="linenos">33</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="add_text_to_concat-35"><a href="#add_text_to_concat-35"><span class="linenos">35</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div> </span></pre></div>
@ -267,17 +331,30 @@ conversions rely on type inference.</p>
</div> </div>
<a class="headerlink" href="#replace_date_funcs"></a> <a class="headerlink" href="#replace_date_funcs"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_date_funcs-36"><a href="#replace_date_funcs-36"><span class="linenos">36</span></a><span class="k">def</span> <span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="replace_date_funcs-38"><a href="#replace_date_funcs-38"><span class="linenos">38</span></a><span class="k">def</span> <span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="replace_date_funcs-37"><a href="#replace_date_funcs-37"><span class="linenos">37</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span> </span><span id="replace_date_funcs-39"><a href="#replace_date_funcs-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="replace_date_funcs-38"><a href="#replace_date_funcs-38"><span class="linenos">38</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span> </span><span id="replace_date_funcs-40"><a href="#replace_date_funcs-40"><span class="linenos">40</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="replace_date_funcs-39"><a href="#replace_date_funcs-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span> </span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
</span><span id="replace_date_funcs-40"><a href="#replace_date_funcs-40"><span class="linenos">40</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span> </span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div> </span></pre></div>
</section>
<section id="COERCIBLE_DATE_OPS">
<div class="attr variable">
<span class="name">COERCIBLE_DATE_OPS</span> =
<input id="COERCIBLE_DATE_OPS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="COERCIBLE_DATE_OPS-view-value"></label><span class="default_value">(&lt;class &#39;<a href="../expressions.html#Add">sqlglot.expressions.Add</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#Sub">sqlglot.expressions.Sub</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NullSafeEQ">sqlglot.expressions.NullSafeEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NullSafeNEQ">sqlglot.expressions.NullSafeNEQ</a>&#39;&gt;)</span>
</div>
<a class="headerlink" href="#COERCIBLE_DATE_OPS"></a>
</section> </section>
<section id="coerce_type"> <section id="coerce_type">
<input id="coerce_type-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="coerce_type-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
@ -290,17 +367,21 @@ conversions rely on type inference.</p>
</div> </div>
<a class="headerlink" href="#coerce_type"></a> <a class="headerlink" href="#coerce_type"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-44"><a href="#coerce_type-44"><span class="linenos">44</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-60"><a href="#coerce_type-60"><span class="linenos">60</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="coerce_type-45"><a href="#coerce_type-45"><span class="linenos">45</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span> </span><span id="coerce_type-61"><a href="#coerce_type-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="coerce_type-46"><a href="#coerce_type-46"><span class="linenos">46</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span> </span><span id="coerce_type-62"><a href="#coerce_type-62"><span class="linenos">62</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="coerce_type-47"><a href="#coerce_type-47"><span class="linenos">47</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span> </span><span id="coerce_type-63"><a href="#coerce_type-63"><span class="linenos">63</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="coerce_type-48"><a href="#coerce_type-48"><span class="linenos">48</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span> </span><span id="coerce_type-64"><a href="#coerce_type-64"><span class="linenos">64</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="coerce_type-49"><a href="#coerce_type-49"><span class="linenos">49</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span> </span><span id="coerce_type-65"><a href="#coerce_type-65"><span class="linenos">65</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="coerce_type-50"><a href="#coerce_type-50"><span class="linenos">50</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span> </span><span id="coerce_type-66"><a href="#coerce_type-66"><span class="linenos">66</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="coerce_type-51"><a href="#coerce_type-51"><span class="linenos">51</span></a> <span class="p">):</span> </span><span id="coerce_type-67"><a href="#coerce_type-67"><span class="linenos">67</span></a> <span class="p">):</span>
</span><span id="coerce_type-52"><a href="#coerce_type-52"><span class="linenos">52</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span> </span><span id="coerce_type-68"><a href="#coerce_type-68"><span class="linenos">68</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="coerce_type-53"><a href="#coerce_type-53"><span class="linenos">53</span></a> </span><span id="coerce_type-69"><a href="#coerce_type-69"><span class="linenos">69</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="coerce_type-54"><a href="#coerce_type-54"><span class="linenos">54</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a>
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div> </span></pre></div>
@ -318,45 +399,49 @@ conversions rely on type inference.</p>
</div> </div>
<a class="headerlink" href="#remove_redundant_casts"></a> <a class="headerlink" href="#remove_redundant_casts"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-57"><a href="#remove_redundant_casts-57"><span class="linenos">57</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-77"><a href="#remove_redundant_casts-77"><span class="linenos">77</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_redundant_casts-58"><a href="#remove_redundant_casts-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="remove_redundant_casts-78"><a href="#remove_redundant_casts-78"><span class="linenos">78</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="remove_redundant_casts-59"><a href="#remove_redundant_casts-59"><span class="linenos">59</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span> </span><span id="remove_redundant_casts-79"><a href="#remove_redundant_casts-79"><span class="linenos">79</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="remove_redundant_casts-60"><a href="#remove_redundant_casts-60"><span class="linenos">60</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span> </span><span id="remove_redundant_casts-80"><a href="#remove_redundant_casts-80"><span class="linenos">80</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-61"><a href="#remove_redundant_casts-61"><span class="linenos">61</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span> </span><span id="remove_redundant_casts-81"><a href="#remove_redundant_casts-81"><span class="linenos">81</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-62"><a href="#remove_redundant_casts-62"><span class="linenos">62</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> </span><span id="remove_redundant_casts-82"><a href="#remove_redundant_casts-82"><span class="linenos">82</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-63"><a href="#remove_redundant_casts-63"><span class="linenos">63</span></a> <span class="p">):</span> </span><span id="remove_redundant_casts-83"><a href="#remove_redundant_casts-83"><span class="linenos">83</span></a> <span class="p">):</span>
</span><span id="remove_redundant_casts-64"><a href="#remove_redundant_casts-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span> </span><span id="remove_redundant_casts-84"><a href="#remove_redundant_casts-84"><span class="linenos">84</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-65"><a href="#remove_redundant_casts-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="remove_redundant_casts-85"><a href="#remove_redundant_casts-85"><span class="linenos">85</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
</section> </section>
<section id="ensure_bool_predicates"> <section id="ensure_bools">
<input id="ensure_bool_predicates-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="ensure_bools-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">ensure_bool_predicates</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span> <span class="name">ensure_bools</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>,</span><span class="param"> <span class="n">replace_func</span><span class="p">:</span> <span class="n">Callable</span><span class="p">[[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
<label class="view-source-button" for="ensure_bool_predicates-view-source"><span>View Source</span></label> <label class="view-source-button" for="ensure_bools-view-source"><span>View Source</span></label>
</div> </div>
<a class="headerlink" href="#ensure_bool_predicates"></a> <a class="headerlink" href="#ensure_bools"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bool_predicates-68"><a href="#ensure_bool_predicates-68"><span class="linenos">68</span></a><span class="k">def</span> <span class="nf">ensure_bool_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-88"><a href="#ensure_bools-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="ensure_bool_predicates-69"><a href="#ensure_bool_predicates-69"><span class="linenos">69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span> </span><span id="ensure_bools-89"><a href="#ensure_bools-89"><span class="linenos"> 89</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="ensure_bool_predicates-70"><a href="#ensure_bool_predicates-70"><span class="linenos">70</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span> </span><span id="ensure_bools-90"><a href="#ensure_bools-90"><span class="linenos"> 90</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="ensure_bool_predicates-71"><a href="#ensure_bool_predicates-71"><span class="linenos">71</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span> </span><span id="ensure_bools-91"><a href="#ensure_bools-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="ensure_bool_predicates-72"><a href="#ensure_bool_predicates-72"><span class="linenos">72</span></a> </span><span id="ensure_bools-92"><a href="#ensure_bools-92"><span class="linenos"> 92</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="ensure_bool_predicates-73"><a href="#ensure_bool_predicates-73"><span class="linenos">73</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">))</span> <span class="ow">or</span> <span class="p">(</span> </span><span id="ensure_bools-93"><a href="#ensure_bools-93"><span class="linenos"> 93</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="ensure_bool_predicates-74"><a href="#ensure_bool_predicates-74"><span class="linenos">74</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span> </span><span id="ensure_bools-94"><a href="#ensure_bools-94"><span class="linenos"> 94</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="ensure_bool_predicates-75"><a href="#ensure_bool_predicates-75"><span class="linenos">75</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> </span><span id="ensure_bools-95"><a href="#ensure_bools-95"><span class="linenos"> 95</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bool_predicates-76"><a href="#ensure_bool_predicates-76"><span class="linenos">76</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span><span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> </span><span id="ensure_bools-96"><a href="#ensure_bools-96"><span class="linenos"> 96</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="ensure_bool_predicates-77"><a href="#ensure_bool_predicates-77"><span class="linenos">77</span></a> <span class="p">):</span> </span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="ensure_bool_predicates-78"><a href="#ensure_bool_predicates-78"><span class="linenos">78</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> </span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="ensure_bool_predicates-79"><a href="#ensure_bool_predicates-79"><span class="linenos">79</span></a> </span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a> <span class="p">):</span>
</span><span id="ensure_bool_predicates-80"><a href="#ensure_bool_predicates-80"><span class="linenos">80</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a>
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -374,12 +459,12 @@ conversions rely on type inference.</p>
</div> </div>
<a class="headerlink" href="#remove_ascending_order"></a> <a class="headerlink" href="#remove_ascending_order"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-83"><a href="#remove_ascending_order-83"><span class="linenos">83</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-107"><a href="#remove_ascending_order-107"><span class="linenos">107</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_ascending_order-84"><a href="#remove_ascending_order-84"><span class="linenos">84</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span> </span><span id="remove_ascending_order-108"><a href="#remove_ascending_order-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="remove_ascending_order-85"><a href="#remove_ascending_order-85"><span class="linenos">85</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span> </span><span id="remove_ascending_order-109"><a href="#remove_ascending_order-109"><span class="linenos">109</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="remove_ascending_order-86"><a href="#remove_ascending_order-86"><span class="linenos">86</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="remove_ascending_order-110"><a href="#remove_ascending_order-110"><span class="linenos">110</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="remove_ascending_order-87"><a href="#remove_ascending_order-87"><span class="linenos">87</span></a> </span><span id="remove_ascending_order-111"><a href="#remove_ascending_order-111"><span class="linenos">111</span></a>
</span><span id="remove_ascending_order-88"><a href="#remove_ascending_order-88"><span class="linenos">88</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="remove_ascending_order-112"><a href="#remove_ascending_order-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>

6
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -256,13 +256,13 @@
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span> </span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> </span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span>
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span> </span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span>
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">&quot;FULL&quot;</span><span class="p">,</span> <span class="s2">&quot;LEFT&quot;</span><span class="p">,</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">}</span> </span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">(</span><span class="s2">&quot;FULL&quot;</span><span class="p">,</span> <span class="s2">&quot;LEFT&quot;</span><span class="p">,</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">)</span>
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="p">)</span> </span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="p">)</span>
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span> </span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> </span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span> </span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span>
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="ow">and</span> <span class="nb">any</span><span class="p">(</span> </span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="ow">and</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">j</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">&quot;FULL&quot;</span><span class="p">,</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">}</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="p">[])</span> </span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">j</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">(</span><span class="s2">&quot;FULL&quot;</span><span class="p">,</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">)</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="p">[])</span>
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">)</span> </span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">)</span>
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="p">)</span> </span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="p">)</span>
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_outer_select_joins_on_inner_select_join</span><span class="p">()</span> </span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_outer_select_joins_on_inner_select_join</span><span class="p">()</span>
@ -568,7 +568,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> = <span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;offset&#39;, &#39;connect&#39;, &#39;pivots&#39;, &#39;having&#39;, &#39;laterals&#39;, &#39;match&#39;, &#39;sample&#39;, &#39;kind&#39;, &#39;qualify&#39;, &#39;sort&#39;, &#39;limit&#39;, &#39;into&#39;, &#39;windows&#39;, &#39;locks&#39;, &#39;cluster&#39;, &#39;settings&#39;, &#39;with&#39;, &#39;distinct&#39;, &#39;group&#39;, &#39;distribute&#39;, &#39;format&#39;}</span> <label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;with&#39;, &#39;kind&#39;, &#39;into&#39;, &#39;offset&#39;, &#39;cluster&#39;, &#39;group&#39;, &#39;having&#39;, &#39;laterals&#39;, &#39;sample&#39;, &#39;match&#39;, &#39;distinct&#39;, &#39;locks&#39;, &#39;sort&#39;, &#39;settings&#39;, &#39;qualify&#39;, &#39;distribute&#39;, &#39;connect&#39;, &#39;format&#39;, &#39;windows&#39;, &#39;limit&#39;, &#39;pivots&#39;}</span>
</div> </div>

58
docs/sqlglot/optimizer/normalize_identifiers.html
View file

@ -60,7 +60,7 @@
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a> </span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span> </span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a> </span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">ParseError</span><span class="p">,</span> <span class="n">exp</span><span class="p">,</span> <span class="n">parse_one</span> </span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span> </span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">DialectType</span> </span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">DialectType</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a>
@ -71,7 +71,7 @@
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a> </span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> </span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="nd">@t</span><span class="o">.</span><span class="n">overload</span> </span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="nd">@t</span><span class="o">.</span><span class="n">overload</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">:</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a> <span class="o">...</span> </span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a> <span class="o">...</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a> </span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a>
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a> </span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
@ -106,21 +106,18 @@
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> Returns:</span> </span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> Returns:</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> The transformed expression.</span> </span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> The transformed expression.</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span> </span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a> <span class="k">try</span><span class="p">:</span> </span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a>
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">parse_one</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span> </span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span> </span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a>
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> </span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span> </span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> </span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span> </span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span> </span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a>
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a>
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -167,21 +164,18 @@
</span><span id="normalize_identifiers-49"><a href="#normalize_identifiers-49"><span class="linenos">49</span></a><span class="sd"> Returns:</span> </span><span id="normalize_identifiers-49"><a href="#normalize_identifiers-49"><span class="linenos">49</span></a><span class="sd"> Returns:</span>
</span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> The transformed expression.</span> </span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> The transformed expression.</span>
</span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span> </span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a> <span class="k">try</span><span class="p">:</span> </span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a>
</span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">parse_one</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span> </span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span> </span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a>
</span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a> </span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span> </span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> </span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span>
</span><span id="normalize_identifiers-60"><a href="#normalize_identifiers-60"><span class="linenos">60</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="normalize_identifiers-60"><a href="#normalize_identifiers-60"><span class="linenos">60</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span> </span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="normalize_identifiers-62"><a href="#normalize_identifiers-62"><span class="linenos">62</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span> </span><span id="normalize_identifiers-62"><a href="#normalize_identifiers-62"><span class="linenos">62</span></a>
</span><span id="normalize_identifiers-63"><a href="#normalize_identifiers-63"><span class="linenos">63</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="normalize_identifiers-63"><a href="#normalize_identifiers-63"><span class="linenos">63</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="normalize_identifiers-64"><a href="#normalize_identifiers-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="normalize_identifiers-65"><a href="#normalize_identifiers-65"><span class="linenos">65</span></a>
</span><span id="normalize_identifiers-66"><a href="#normalize_identifiers-66"><span class="linenos">66</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span></pre></div> </span></pre></div>

10
docs/sqlglot/optimizer/optimizer.html
View file

File diff suppressed because one or more lines are too long

1210
docs/sqlglot/optimizer/qualify_columns.html
View file

File diff suppressed because it is too large Load diff

390
docs/sqlglot/optimizer/qualify_tables.html
View file

@ -56,106 +56,113 @@
<label class="view-source-button" for="mod-qualify_tables-view-source"><span>View Source</span></label> <label class="view-source-button" for="mod-qualify_tables-view-source"><span>View Source</span></label>
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">import</span> <span class="nn">itertools</span> <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">annotations</span>
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span> </span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a> </span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">itertools</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">alias</span><span class="p">,</span> <span class="n">exp</span> </span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span> </span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">csv_reader</span><span class="p">,</span> <span class="n">name_sequence</span> </span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">alias</span><span class="p">,</span> <span class="n">exp</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span> </span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">DialectType</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a> </span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">csv_reader</span><span class="p">,</span> <span class="n">name_sequence</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a> </span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span> </span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> </span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span> </span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span> </span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> </span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="sd"> Examples:</span> </span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span> </span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span> </span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span> </span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> Examples:</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> &gt;&gt;&gt;</span> </span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span> </span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span> </span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span> </span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> </span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> Args:</span> </span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> expression: Expression to qualify</span> </span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> db: Database name</span> </span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> catalog: Catalog name</span> </span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> schema: A schema to populate</span> </span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> Args:</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> </span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd"> Returns:</span> </span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd"> db: Database name</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> The qualified expression.</span> </span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span> </span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> </span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span> </span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> Returns:</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span> </span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> The qualified expression.</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span> </span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span> </span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span> </span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> </span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span> </span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> </span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span> </span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span> </span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">))</span> </span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">):</span> </span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">))</span> </span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> </span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span> </span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> </span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span> </span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span> </span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="p">)</span> </span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> </span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span> </span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span> </span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span> </span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">source</span><span class="p">,</span> </span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="p">)</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span> </span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="p">)</span> </span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span> </span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span> </span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span> </span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="p">)</span> </span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span> </span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> </span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="p">)</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> </span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span> </span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span> </span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span> </span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">)</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> </span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -165,102 +172,106 @@
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">qualify_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="o">~</span><span class="n">E</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="o">~</span><span class="n">E</span>:</span></span> <span class="name">qualify_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="o">~</span><span class="n">E</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="o">~</span><span class="n">E</span>:</span></span>
<label class="view-source-button" for="qualify_tables-view-source"><span>View Source</span></label> <label class="view-source-button" for="qualify_tables-view-source"><span>View Source</span></label>
</div> </div>
<a class="headerlink" href="#qualify_tables"></a> <a class="headerlink" href="#qualify_tables"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify_tables-12"><a href="#qualify_tables-12"><span class="linenos"> 12</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span> <div class="pdoc-code codehilite"><pre><span></span><span id="qualify_tables-15"><a href="#qualify_tables-15"><span class="linenos"> 15</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span>
</span><span id="qualify_tables-13"><a href="#qualify_tables-13"><span class="linenos"> 13</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> </span><span id="qualify_tables-16"><a href="#qualify_tables-16"><span class="linenos"> 16</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
</span><span id="qualify_tables-14"><a href="#qualify_tables-14"><span class="linenos"> 14</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-17"><a href="#qualify_tables-17"><span class="linenos"> 17</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-15"><a href="#qualify_tables-15"><span class="linenos"> 15</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-18"><a href="#qualify_tables-18"><span class="linenos"> 18</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-16"><a href="#qualify_tables-16"><span class="linenos"> 16</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-17"><a href="#qualify_tables-17"><span class="linenos"> 17</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-18"><a href="#qualify_tables-18"><span class="linenos"> 18</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span> </span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span> </span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a> </span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a><span class="sd"> Examples:</span> </span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a>
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> Examples:</span>
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span> </span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span> </span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span> </span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> &gt;&gt;&gt;</span> </span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span> </span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span> </span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span> </span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a> </span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> Args:</span> </span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a>
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> expression: Expression to qualify</span> </span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> Args:</span>
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> db: Database name</span> </span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> catalog: Catalog name</span> </span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a><span class="sd"> db: Database name</span>
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> schema: A schema to populate</span> </span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a> </span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a><span class="sd"> Returns:</span> </span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a> </span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a>
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span> </span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> Returns:</span>
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span> </span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span> </span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span> </span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a>
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span> </span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> </span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span> </span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span> </span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span> </span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span> </span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> </span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a>
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span> </span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> </span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span> </span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a>
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span> </span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">))</span> </span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">):</span> </span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">))</span> </span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a>
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> </span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span> </span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> </span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span> </span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a>
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span> </span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="p">)</span> </span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> </span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span> </span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a>
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span> </span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span> </span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="n">source</span><span class="p">,</span> </span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="p">)</span>
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span> </span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a>
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="p">)</span> </span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span> </span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span> </span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span> </span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span> </span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="p">)</span> </span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span> </span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> </span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> <span class="p">)</span>
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> </span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span> </span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span> </span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span> </span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="p">)</span>
</span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a> </span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a>
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a>
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -290,6 +301,7 @@
<li><strong>db:</strong> Database name</li> <li><strong>db:</strong> Database name</li>
<li><strong>catalog:</strong> Catalog name</li> <li><strong>catalog:</strong> Catalog name</li>
<li><strong>schema:</strong> A schema to populate</li> <li><strong>schema:</strong> A schema to populate</li>
<li><strong>dialect:</strong> The dialect to parse catalog and schema into.</li>
</ul> </ul>
<h6 id="returns">Returns:</h6> <h6 id="returns">Returns:</h6>

2926
docs/sqlglot/optimizer/scope.html
View file

File diff suppressed because it is too large Load diff

1870
docs/sqlglot/optimizer/simplify.html
View file

File diff suppressed because it is too large Load diff

21731
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2658
docs/sqlglot/schema.html
View file

File diff suppressed because it is too large Load diff

1264
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

5057
docs/sqlglot/tokens.html
View file

File diff suppressed because one or more lines are too long

1060
docs/sqlglot/transforms.html
View file

File diff suppressed because it is too large Load diff

9
sqlglot/__init__.py
View file

@ -22,6 +22,7 @@ from sqlglot.expressions import (
Expression as Expression, Expression as Expression,
alias_ as alias, alias_ as alias,
and_ as and_, and_ as and_,
case as case,
cast as cast, cast as cast,
column as column, column as column,
condition as condition, condition as condition,
@ -82,8 +83,7 @@ def parse(
Returns: Returns:
The resulting syntax tree collection. The resulting syntax tree collection.
""" """
dialect = Dialect.get_or_raise(read or dialect)() return Dialect.get_or_raise(read or dialect).parse(sql, **opts)
return dialect.parse(sql, **opts)
@t.overload @t.overload
@ -117,7 +117,7 @@ def parse_one(
The syntax tree for the first parsed statement. The syntax tree for the first parsed statement.
""" """
dialect = Dialect.get_or_raise(read or dialect)() dialect = Dialect.get_or_raise(read or dialect)
if into: if into:
result = dialect.parse_into(into, sql, **opts) result = dialect.parse_into(into, sql, **opts)
@ -157,7 +157,8 @@ def transpile(
The list of transpiled SQL statements. The list of transpiled SQL statements.
""" """
write = (read if write is None else write) if identity else write write = (read if write is None else write) if identity else write
write = Dialect.get_or_raise(write)
return [ return [
Dialect.get_or_raise(write)().generate(expression, copy=False, **opts) if expression else "" write.generate(expression, copy=False, **opts) if expression else ""
for expression in parse(sql, read, error_level=error_level) for expression in parse(sql, read, error_level=error_level)
] ]

2
sqlglot/__main__.py
View file

@ -81,7 +81,7 @@ if args.parse:
) )
] ]
elif args.tokenize: elif args.tokenize:
objs = sqlglot.Dialect.get_or_raise(args.read)().tokenize(sql) objs = sqlglot.Dialect.get_or_raise(args.read).tokenize(sql)
else: else:
objs = sqlglot.transpile( objs = sqlglot.transpile(
sql, sql,

39
sqlglot/dataframe/sql/dataframe.py
View file

@ -297,27 +297,26 @@ class DataFrame:
select_expressions.append(expression_select_pair) # type: ignore select_expressions.append(expression_select_pair) # type: ignore
return select_expressions return select_expressions
def sql( def sql(self, dialect: DialectType = None, optimize: bool = True, **kwargs) -> t.List[str]:
self, dialect: t.Optional[DialectType] = None, optimize: bool = True, **kwargs
) -> t.List[str]:
from sqlglot.dataframe.sql.session import SparkSession from sqlglot.dataframe.sql.session import SparkSession
if dialect and Dialect.get_or_raise(dialect)() != SparkSession().dialect: dialect = Dialect.get_or_raise(dialect or SparkSession().dialect)
logger.warning(
f"The recommended way of defining a dialect is by doing `SparkSession.builder.config('sqlframe.dialect', '{dialect}').getOrCreate()`. It is no longer needed then when calling `sql`. If you run into issues try updating your query to use this pattern."
)
df = self._resolve_pending_hints() df = self._resolve_pending_hints()
select_expressions = df._get_select_expressions() select_expressions = df._get_select_expressions()
output_expressions: t.List[t.Union[exp.Select, exp.Cache, exp.Drop]] = [] output_expressions: t.List[t.Union[exp.Select, exp.Cache, exp.Drop]] = []
replacement_mapping: t.Dict[exp.Identifier, exp.Identifier] = {} replacement_mapping: t.Dict[exp.Identifier, exp.Identifier] = {}
for expression_type, select_expression in select_expressions: for expression_type, select_expression in select_expressions:
select_expression = select_expression.transform(replace_id_value, replacement_mapping) select_expression = select_expression.transform(replace_id_value, replacement_mapping)
if optimize: if optimize:
quote_identifiers(select_expression) quote_identifiers(select_expression, dialect=dialect)
select_expression = t.cast( select_expression = t.cast(
exp.Select, optimize_func(select_expression, dialect=SparkSession().dialect) exp.Select, optimize_func(select_expression, dialect=dialect)
) )
select_expression = df._replace_cte_names_with_hashes(select_expression) select_expression = df._replace_cte_names_with_hashes(select_expression)
expression: t.Union[exp.Select, exp.Cache, exp.Drop] expression: t.Union[exp.Select, exp.Cache, exp.Drop]
if expression_type == exp.Cache: if expression_type == exp.Cache:
cache_table_name = df._create_hash_from_expression(select_expression) cache_table_name = df._create_hash_from_expression(select_expression)
@ -330,13 +329,12 @@ class DataFrame:
sqlglot.schema.add_table( sqlglot.schema.add_table(
cache_table_name, cache_table_name,
{ {
expression.alias_or_name: expression.type.sql( expression.alias_or_name: expression.type.sql(dialect=dialect)
dialect=SparkSession().dialect
)
for expression in select_expression.expressions for expression in select_expression.expressions
}, },
dialect=SparkSession().dialect, dialect=dialect,
) )
cache_storage_level = select_expression.args["cache_storage_level"] cache_storage_level = select_expression.args["cache_storage_level"]
options = [ options = [
exp.Literal.string("storageLevel"), exp.Literal.string("storageLevel"),
@ -345,6 +343,7 @@ class DataFrame:
expression = exp.Cache( expression = exp.Cache(
this=cache_table, expression=select_expression, lazy=True, options=options this=cache_table, expression=select_expression, lazy=True, options=options
) )
# We will drop the "view" if it exists before running the cache table # We will drop the "view" if it exists before running the cache table
output_expressions.append(exp.Drop(this=cache_table, exists=True, kind="VIEW")) output_expressions.append(exp.Drop(this=cache_table, exists=True, kind="VIEW"))
elif expression_type == exp.Create: elif expression_type == exp.Create:
@ -355,18 +354,17 @@ class DataFrame:
select_without_ctes = select_expression.copy() select_without_ctes = select_expression.copy()
select_without_ctes.set("with", None) select_without_ctes.set("with", None)
expression.set("expression", select_without_ctes) expression.set("expression", select_without_ctes)
if select_expression.ctes: if select_expression.ctes:
expression.set("with", exp.With(expressions=select_expression.ctes)) expression.set("with", exp.With(expressions=select_expression.ctes))
elif expression_type == exp.Select: elif expression_type == exp.Select:
expression = select_expression expression = select_expression
else: else:
raise ValueError(f"Invalid expression type: {expression_type}") raise ValueError(f"Invalid expression type: {expression_type}")
output_expressions.append(expression) output_expressions.append(expression)
return [ return [expression.sql(dialect=dialect, **kwargs) for expression in output_expressions]
expression.sql(**{"dialect": SparkSession().dialect, **kwargs})
for expression in output_expressions
]
def copy(self, **kwargs) -> DataFrame: def copy(self, **kwargs) -> DataFrame:
return DataFrame(**object_to_dict(self, **kwargs)) return DataFrame(**object_to_dict(self, **kwargs))
@ -542,12 +540,7 @@ class DataFrame:
""" """
columns = self._ensure_and_normalize_cols(cols) columns = self._ensure_and_normalize_cols(cols)
pre_ordered_col_indexes = [ pre_ordered_col_indexes = [
x i for i, col in enumerate(columns) if isinstance(col.expression, exp.Ordered)
for x in [
i if isinstance(col.expression, exp.Ordered) else None
for i, col in enumerate(columns)
]
if x is not None
] ]
if ascending is None: if ascending is None:
ascending = [True] * len(columns) ascending = [True] * len(columns)

2
sqlglot/dataframe/sql/functions.py
View file

@ -306,7 +306,7 @@ def collect_list(col: ColumnOrName) -> Column:
def collect_set(col: ColumnOrName) -> Column: def collect_set(col: ColumnOrName) -> Column:
return Column.invoke_expression_over_column(col, expression.SetAgg) return Column.invoke_expression_over_column(col, expression.ArrayUniqueAgg)
def hypot(col1: t.Union[ColumnOrName, float], col2: t.Union[ColumnOrName, float]) -> Column: def hypot(col1: t.Union[ColumnOrName, float], col2: t.Union[ColumnOrName, float]) -> Column:

4
sqlglot/dataframe/sql/session.py
View file

@ -28,7 +28,7 @@ class SparkSession:
self.known_sequence_ids = set() self.known_sequence_ids = set()
self.name_to_sequence_id_mapping = defaultdict(list) self.name_to_sequence_id_mapping = defaultdict(list)
self.incrementing_id = 1 self.incrementing_id = 1
self.dialect = Dialect.get_or_raise(self.DEFAULT_DIALECT)() self.dialect = Dialect.get_or_raise(self.DEFAULT_DIALECT)
def __new__(cls, *args, **kwargs) -> SparkSession: def __new__(cls, *args, **kwargs) -> SparkSession:
if cls._instance is None: if cls._instance is None:
@ -182,7 +182,7 @@ class SparkSession:
def getOrCreate(self) -> SparkSession: def getOrCreate(self) -> SparkSession:
spark = SparkSession() spark = SparkSession()
spark.dialect = Dialect.get_or_raise(self.dialect)() spark.dialect = Dialect.get_or_raise(self.dialect)
return spark return spark
@classproperty @classproperty

117
sqlglot/dialects/bigquery.py
View file

@ -8,6 +8,7 @@ from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
arg_max_or_min_no_count, arg_max_or_min_no_count,
binary_from_function, binary_from_function,
date_add_interval_sql, date_add_interval_sql,
@ -23,6 +24,7 @@ from sqlglot.dialects.dialect import (
regexp_replace_sql, regexp_replace_sql,
rename_func, rename_func,
timestrtotime_sql, timestrtotime_sql,
ts_or_ds_add_cast,
ts_or_ds_to_date_sql, ts_or_ds_to_date_sql,
) )
from sqlglot.helper import seq_get, split_num_words from sqlglot.helper import seq_get, split_num_words
@ -174,6 +176,44 @@ def _parse_to_hex(args: t.List) -> exp.Hex | exp.MD5:
return exp.MD5(this=arg.this) if isinstance(arg, exp.MD5Digest) else exp.Hex(this=arg) return exp.MD5(this=arg.this) if isinstance(arg, exp.MD5Digest) else exp.Hex(this=arg)
def _array_contains_sql(self: BigQuery.Generator, expression: exp.ArrayContains) -> str:
return self.sql(
exp.Exists(
this=exp.select("1")
.from_(exp.Unnest(expressions=[expression.left]).as_("_unnest", table=["_col"]))
.where(exp.column("_col").eq(expression.right))
)
)
def _ts_or_ds_add_sql(self: BigQuery.Generator, expression: exp.TsOrDsAdd) -> str:
return date_add_interval_sql("DATE", "ADD")(self, ts_or_ds_add_cast(expression))
def _ts_or_ds_diff_sql(self: BigQuery.Generator, expression: exp.TsOrDsDiff) -> str:
expression.this.replace(exp.cast(expression.this, "TIMESTAMP", copy=True))
expression.expression.replace(exp.cast(expression.expression, "TIMESTAMP", copy=True))
unit = expression.args.get("unit") or "DAY"
return self.func("DATE_DIFF", expression.this, expression.expression, unit)
def _unix_to_time_sql(self: BigQuery.Generator, expression: exp.UnixToTime) -> str:
scale = expression.args.get("scale")
timestamp = self.sql(expression, "this")
if scale in (None, exp.UnixToTime.SECONDS):
return f"TIMESTAMP_SECONDS({timestamp})"
if scale == exp.UnixToTime.MILLIS:
return f"TIMESTAMP_MILLIS({timestamp})"
if scale == exp.UnixToTime.MICROS:
return f"TIMESTAMP_MICROS({timestamp})"
if scale == exp.UnixToTime.NANOS:
# We need to cast to INT64 because that's what BQ expects
return f"TIMESTAMP_MICROS(CAST({timestamp} / 1000 AS INT64))"
self.unsupported(f"Unsupported scale for timestamp: {scale}.")
return ""
class BigQuery(Dialect): class BigQuery(Dialect):
UNNEST_COLUMN_ONLY = True UNNEST_COLUMN_ONLY = True
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
@ -181,7 +221,7 @@ class BigQuery(Dialect):
LOG_BASE_FIRST = False LOG_BASE_FIRST = False
# https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#case_sensitivity # https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#case_sensitivity
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
# bigquery udfs are case sensitive # bigquery udfs are case sensitive
NORMALIZE_FUNCTIONS = False NORMALIZE_FUNCTIONS = False
@ -220,8 +260,7 @@ class BigQuery(Dialect):
# https://cloud.google.com/bigquery/docs/querying-partitioned-tables#query_an_ingestion-time_partitioned_table # https://cloud.google.com/bigquery/docs/querying-partitioned-tables#query_an_ingestion-time_partitioned_table
PSEUDOCOLUMNS = {"_PARTITIONTIME", "_PARTITIONDATE"} PSEUDOCOLUMNS = {"_PARTITIONTIME", "_PARTITIONDATE"}
@classmethod def normalize_identifier(self, expression: E) -> E:
def normalize_identifier(cls, expression: E) -> E:
if isinstance(expression, exp.Identifier): if isinstance(expression, exp.Identifier):
parent = expression.parent parent = expression.parent
while isinstance(parent, exp.Dot): while isinstance(parent, exp.Dot):
@ -265,7 +304,6 @@ class BigQuery(Dialect):
"DECLARE": TokenType.COMMAND, "DECLARE": TokenType.COMMAND,
"FLOAT64": TokenType.DOUBLE, "FLOAT64": TokenType.DOUBLE,
"FOR SYSTEM_TIME": TokenType.TIMESTAMP_SNAPSHOT, "FOR SYSTEM_TIME": TokenType.TIMESTAMP_SNAPSHOT,
"INT64": TokenType.BIGINT,
"MODEL": TokenType.MODEL, "MODEL": TokenType.MODEL,
"NOT DETERMINISTIC": TokenType.VOLATILE, "NOT DETERMINISTIC": TokenType.VOLATILE,
"RECORD": TokenType.STRUCT, "RECORD": TokenType.STRUCT,
@ -316,6 +354,15 @@ class BigQuery(Dialect):
"TIME_SUB": parse_date_delta_with_interval(exp.TimeSub), "TIME_SUB": parse_date_delta_with_interval(exp.TimeSub),
"TIMESTAMP_ADD": parse_date_delta_with_interval(exp.TimestampAdd), "TIMESTAMP_ADD": parse_date_delta_with_interval(exp.TimestampAdd),
"TIMESTAMP_SUB": parse_date_delta_with_interval(exp.TimestampSub), "TIMESTAMP_SUB": parse_date_delta_with_interval(exp.TimestampSub),
"TIMESTAMP_MICROS": lambda args: exp.UnixToTime(
this=seq_get(args, 0), scale=exp.UnixToTime.MICROS
),
"TIMESTAMP_MILLIS": lambda args: exp.UnixToTime(
this=seq_get(args, 0), scale=exp.UnixToTime.MILLIS
),
"TIMESTAMP_SECONDS": lambda args: exp.UnixToTime(
this=seq_get(args, 0), scale=exp.UnixToTime.SECONDS
),
"TO_JSON_STRING": exp.JSONFormat.from_arg_list, "TO_JSON_STRING": exp.JSONFormat.from_arg_list,
} }
@ -358,6 +405,24 @@ class BigQuery(Dialect):
NULL_TOKENS = {TokenType.NULL, TokenType.UNKNOWN} NULL_TOKENS = {TokenType.NULL, TokenType.UNKNOWN}
STATEMENT_PARSERS = {
**parser.Parser.STATEMENT_PARSERS,
TokenType.END: lambda self: self._parse_as_command(self._prev),
TokenType.FOR: lambda self: self._parse_for_in(),
}
BRACKET_OFFSETS = {
"OFFSET": (0, False),
"ORDINAL": (1, False),
"SAFE_OFFSET": (0, True),
"SAFE_ORDINAL": (1, True),
}
def _parse_for_in(self) -> exp.ForIn:
this = self._parse_range()
self._match_text_seq("DO")
return self.expression(exp.ForIn, this=this, expression=self._parse_statement())
def _parse_table_part(self, schema: bool = False) -> t.Optional[exp.Expression]: def _parse_table_part(self, schema: bool = False) -> t.Optional[exp.Expression]:
this = super()._parse_table_part(schema=schema) or self._parse_number() this = super()._parse_table_part(schema=schema) or self._parse_number()
@ -419,6 +484,26 @@ class BigQuery(Dialect):
return json_object return json_object
def _parse_bracket(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
bracket = super()._parse_bracket(this)
if this is bracket:
return bracket
if isinstance(bracket, exp.Bracket):
for expression in bracket.expressions:
name = expression.name.upper()
if name not in self.BRACKET_OFFSETS:
break
offset, safe = self.BRACKET_OFFSETS[name]
bracket.set("offset", offset)
bracket.set("safe", safe)
expression.replace(expression.expressions[0])
return bracket
class Generator(generator.Generator): class Generator(generator.Generator):
EXPLICIT_UNION = True EXPLICIT_UNION = True
INTERVAL_ALLOWS_PLURAL_FORM = False INTERVAL_ALLOWS_PLURAL_FORM = False
@ -430,12 +515,14 @@ class BigQuery(Dialect):
NVL2_SUPPORTED = False NVL2_SUPPORTED = False
UNNEST_WITH_ORDINALITY = False UNNEST_WITH_ORDINALITY = False
COLLATE_IS_FUNC = True COLLATE_IS_FUNC = True
LIMIT_ONLY_LITERALS = True
TRANSFORMS = { TRANSFORMS = {
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,
exp.ApproxDistinct: rename_func("APPROX_COUNT_DISTINCT"), exp.ApproxDistinct: rename_func("APPROX_COUNT_DISTINCT"),
exp.ArgMax: arg_max_or_min_no_count("MAX_BY"), exp.ArgMax: arg_max_or_min_no_count("MAX_BY"),
exp.ArgMin: arg_max_or_min_no_count("MIN_BY"), exp.ArgMin: arg_max_or_min_no_count("MIN_BY"),
exp.ArrayContains: _array_contains_sql,
exp.ArraySize: rename_func("ARRAY_LENGTH"), exp.ArraySize: rename_func("ARRAY_LENGTH"),
exp.Cast: transforms.preprocess([transforms.remove_precision_parameterized_types]), exp.Cast: transforms.preprocess([transforms.remove_precision_parameterized_types]),
exp.CollateProperty: lambda self, e: f"DEFAULT COLLATE {self.sql(e, 'this')}" exp.CollateProperty: lambda self, e: f"DEFAULT COLLATE {self.sql(e, 'this')}"
@ -498,10 +585,13 @@ class BigQuery(Dialect):
exp.TimestampAdd: date_add_interval_sql("TIMESTAMP", "ADD"), exp.TimestampAdd: date_add_interval_sql("TIMESTAMP", "ADD"),
exp.TimestampSub: date_add_interval_sql("TIMESTAMP", "SUB"), exp.TimestampSub: date_add_interval_sql("TIMESTAMP", "SUB"),
exp.TimeStrToTime: timestrtotime_sql, exp.TimeStrToTime: timestrtotime_sql,
exp.TimeToStr: lambda self, e: f"FORMAT_DATE({self.format_time(e)}, {self.sql(e, 'this')})",
exp.Trim: lambda self, e: self.func(f"TRIM", e.this, e.expression), exp.Trim: lambda self, e: self.func(f"TRIM", e.this, e.expression),
exp.TsOrDsAdd: date_add_interval_sql("DATE", "ADD"), exp.TsOrDsAdd: _ts_or_ds_add_sql,
exp.TsOrDsDiff: _ts_or_ds_diff_sql,
exp.TsOrDsToDate: ts_or_ds_to_date_sql("bigquery"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("bigquery"),
exp.Unhex: rename_func("FROM_HEX"), exp.Unhex: rename_func("FROM_HEX"),
exp.UnixToTime: _unix_to_time_sql,
exp.Values: _derived_table_values_to_unnest, exp.Values: _derived_table_values_to_unnest,
exp.VariancePop: rename_func("VAR_POP"), exp.VariancePop: rename_func("VAR_POP"),
} }
@ -671,6 +761,23 @@ class BigQuery(Dialect):
return inline_array_sql(self, expression) return inline_array_sql(self, expression)
def bracket_sql(self, expression: exp.Bracket) -> str:
expressions = expression.expressions
expressions_sql = ", ".join(self.sql(e) for e in expressions)
offset = expression.args.get("offset")
if offset == 0:
expressions_sql = f"OFFSET({expressions_sql})"
elif offset == 1:
expressions_sql = f"ORDINAL({expressions_sql})"
else:
self.unsupported(f"Unsupported array offset: {offset}")
if expression.args.get("safe"):
expressions_sql = f"SAFE_{expressions_sql}"
return f"{self.sql(expression, 'this')}[{expressions_sql}]"
def transaction_sql(self, *_) -> str: def transaction_sql(self, *_) -> str:
return "BEGIN TRANSACTION" return "BEGIN TRANSACTION"

36
sqlglot/dialects/clickhouse.py
View file

@ -35,8 +35,8 @@ def _quantile_sql(self, e):
class ClickHouse(Dialect): class ClickHouse(Dialect):
NORMALIZE_FUNCTIONS: bool | str = False NORMALIZE_FUNCTIONS: bool | str = False
NULL_ORDERING = "nulls_are_last" NULL_ORDERING = "nulls_are_last"
STRICT_STRING_CONCAT = True
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
SAFE_DIVISION = True
ESCAPE_SEQUENCES = { ESCAPE_SEQUENCES = {
"\\0": "\0", "\\0": "\0",
@ -63,11 +63,7 @@ class ClickHouse(Dialect):
"FLOAT32": TokenType.FLOAT, "FLOAT32": TokenType.FLOAT,
"FLOAT64": TokenType.DOUBLE, "FLOAT64": TokenType.DOUBLE,
"GLOBAL": TokenType.GLOBAL, "GLOBAL": TokenType.GLOBAL,
"INT16": TokenType.SMALLINT,
"INT256": TokenType.INT256, "INT256": TokenType.INT256,
"INT32": TokenType.INT,
"INT64": TokenType.BIGINT,
"INT8": TokenType.TINYINT,
"LOWCARDINALITY": TokenType.LOWCARDINALITY, "LOWCARDINALITY": TokenType.LOWCARDINALITY,
"MAP": TokenType.MAP, "MAP": TokenType.MAP,
"NESTED": TokenType.NESTED, "NESTED": TokenType.NESTED,
@ -112,6 +108,7 @@ class ClickHouse(Dialect):
FUNCTION_PARSERS = { FUNCTION_PARSERS = {
**parser.Parser.FUNCTION_PARSERS, **parser.Parser.FUNCTION_PARSERS,
"ARRAYJOIN": lambda self: self.expression(exp.Explode, this=self._parse_expression()),
"QUANTILE": lambda self: self._parse_quantile(), "QUANTILE": lambda self: self._parse_quantile(),
} }
@ -223,12 +220,13 @@ class ClickHouse(Dialect):
except ParseError: except ParseError:
# WITH <expression> AS <identifier> # WITH <expression> AS <identifier>
self._retreat(index) self._retreat(index)
statement = self._parse_statement()
if statement and isinstance(statement.this, exp.Alias): return self.expression(
self.raise_error("Expected CTE to have alias") exp.CTE,
this=self._parse_field(),
return self.expression(exp.CTE, this=statement, alias=statement and statement.this) alias=self._parse_table_alias(),
scalar=True,
)
def _parse_join_parts( def _parse_join_parts(
self, self,
@ -385,9 +383,11 @@ class ClickHouse(Dialect):
exp.DateDiff: lambda self, e: self.func( exp.DateDiff: lambda self, e: self.func(
"DATE_DIFF", exp.Literal.string(e.text("unit") or "day"), e.expression, e.this "DATE_DIFF", exp.Literal.string(e.text("unit") or "day"), e.expression, e.this
), ),
exp.Explode: rename_func("arrayJoin"),
exp.Final: lambda self, e: f"{self.sql(e, 'this')} FINAL", exp.Final: lambda self, e: f"{self.sql(e, 'this')} FINAL",
exp.IsNan: rename_func("isNaN"), exp.IsNan: rename_func("isNaN"),
exp.Map: lambda self, e: _lower_func(var_map_sql(self, e)), exp.Map: lambda self, e: _lower_func(var_map_sql(self, e)),
exp.Nullif: rename_func("nullIf"),
exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}", exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}",
exp.Pivot: no_pivot_sql, exp.Pivot: no_pivot_sql,
exp.Quantile: _quantile_sql, exp.Quantile: _quantile_sql,
@ -459,19 +459,11 @@ class ClickHouse(Dialect):
return super().datatype_sql(expression) return super().datatype_sql(expression)
def safeconcat_sql(self, expression: exp.SafeConcat) -> str:
# Clickhouse errors out if we try to cast a NULL value to TEXT
return self.func(
"CONCAT",
*[
exp.func("if", e.is_(exp.null()), e, exp.cast(e, "text"))
for e in t.cast(t.List[exp.Condition], expression.expressions)
],
)
def cte_sql(self, expression: exp.CTE) -> str: def cte_sql(self, expression: exp.CTE) -> str:
if isinstance(expression.this, exp.Alias): if expression.args.get("scalar"):
return self.sql(expression, "this") this = self.sql(expression, "this")
alias = self.sql(expression, "alias")
return f"{this} AS {alias}"
return super().cte_sql(expression) return super().cte_sql(expression)

13
sqlglot/dialects/databricks.py
View file

@ -1,13 +1,18 @@
from __future__ import annotations from __future__ import annotations
from sqlglot import exp, transforms from sqlglot import exp, transforms
from sqlglot.dialects.dialect import parse_date_delta, timestamptrunc_sql from sqlglot.dialects.dialect import (
date_delta_sql,
parse_date_delta,
timestamptrunc_sql,
)
from sqlglot.dialects.spark import Spark from sqlglot.dialects.spark import Spark
from sqlglot.dialects.tsql import generate_date_delta_with_unit_sql
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
class Databricks(Spark): class Databricks(Spark):
SAFE_DIVISION = False
class Parser(Spark.Parser): class Parser(Spark.Parser):
LOG_DEFAULTS_TO_LN = True LOG_DEFAULTS_TO_LN = True
STRICT_CAST = True STRICT_CAST = True
@ -27,8 +32,8 @@ class Databricks(Spark):
class Generator(Spark.Generator): class Generator(Spark.Generator):
TRANSFORMS = { TRANSFORMS = {
**Spark.Generator.TRANSFORMS, **Spark.Generator.TRANSFORMS,
exp.DateAdd: generate_date_delta_with_unit_sql, exp.DateAdd: date_delta_sql("DATEADD"),
exp.DateDiff: generate_date_delta_with_unit_sql, exp.DateDiff: date_delta_sql("DATEDIFF"),
exp.DatetimeAdd: lambda self, e: self.func( exp.DatetimeAdd: lambda self, e: self.func(
"TIMESTAMPADD", e.text("unit"), e.expression, e.this "TIMESTAMPADD", e.text("unit"), e.expression, e.this
), ),

218
sqlglot/dialects/dialect.py
View file

@ -1,14 +1,14 @@
from __future__ import annotations from __future__ import annotations
import typing as t import typing as t
from enum import Enum from enum import Enum, auto
from functools import reduce from functools import reduce
from sqlglot import exp from sqlglot import exp
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.errors import ParseError from sqlglot.errors import ParseError
from sqlglot.generator import Generator from sqlglot.generator import Generator
from sqlglot.helper import flatten, seq_get from sqlglot.helper import AutoName, flatten, seq_get
from sqlglot.parser import Parser from sqlglot.parser import Parser
from sqlglot.time import TIMEZONES, format_time from sqlglot.time import TIMEZONES, format_time
from sqlglot.tokens import Token, Tokenizer, TokenType from sqlglot.tokens import Token, Tokenizer, TokenType
@ -16,6 +16,9 @@ from sqlglot.trie import new_trie
B = t.TypeVar("B", bound=exp.Binary) B = t.TypeVar("B", bound=exp.Binary)
DATE_ADD_OR_DIFF = t.Union[exp.DateAdd, exp.TsOrDsAdd, exp.DateDiff, exp.TsOrDsDiff]
DATE_ADD_OR_SUB = t.Union[exp.DateAdd, exp.TsOrDsAdd, exp.DateSub]
class Dialects(str, Enum): class Dialects(str, Enum):
DIALECT = "" DIALECT = ""
@ -43,6 +46,15 @@ class Dialects(str, Enum):
Doris = "doris" Doris = "doris"
class NormalizationStrategy(str, AutoName):
"""Specifies the strategy according to which identifiers should be normalized."""
LOWERCASE = auto() # Unquoted identifiers are lowercased
UPPERCASE = auto() # Unquoted identifiers are uppercased
CASE_SENSITIVE = auto() # Always case-sensitive, regardless of quotes
CASE_INSENSITIVE = auto() # Always case-insensitive, regardless of quotes
class _Dialect(type): class _Dialect(type):
classes: t.Dict[str, t.Type[Dialect]] = {} classes: t.Dict[str, t.Type[Dialect]] = {}
@ -106,26 +118,8 @@ class _Dialect(type):
klass.HEX_START, klass.HEX_END = get_start_end(TokenType.HEX_STRING) klass.HEX_START, klass.HEX_END = get_start_end(TokenType.HEX_STRING)
klass.BYTE_START, klass.BYTE_END = get_start_end(TokenType.BYTE_STRING) klass.BYTE_START, klass.BYTE_END = get_start_end(TokenType.BYTE_STRING)
dialect_properties = {
**{
k: v
for k, v in vars(klass).items()
if not callable(v) and not isinstance(v, classmethod) and not k.startswith("__")
},
"TOKENIZER_CLASS": klass.tokenizer_class,
}
if enum not in ("", "bigquery"): if enum not in ("", "bigquery"):
dialect_properties["SELECT_KINDS"] = () klass.generator_class.SELECT_KINDS = ()
# Pass required dialect properties to the tokenizer, parser and generator classes
for subclass in (klass.tokenizer_class, klass.parser_class, klass.generator_class):
for name, value in dialect_properties.items():
if hasattr(subclass, name):
setattr(subclass, name, value)
if not klass.STRICT_STRING_CONCAT and klass.DPIPE_IS_STRING_CONCAT:
klass.parser_class.BITWISE[TokenType.DPIPE] = exp.SafeDPipe
if not klass.SUPPORTS_SEMI_ANTI_JOIN: if not klass.SUPPORTS_SEMI_ANTI_JOIN:
klass.parser_class.TABLE_ALIAS_TOKENS = klass.parser_class.TABLE_ALIAS_TOKENS | { klass.parser_class.TABLE_ALIAS_TOKENS = klass.parser_class.TABLE_ALIAS_TOKENS | {
@ -133,8 +127,6 @@ class _Dialect(type):
TokenType.SEMI, TokenType.SEMI,
} }
klass.generator_class.can_identify = klass.can_identify
return klass return klass
@ -148,9 +140,8 @@ class Dialect(metaclass=_Dialect):
# Determines whether or not the table alias comes after tablesample # Determines whether or not the table alias comes after tablesample
ALIAS_POST_TABLESAMPLE = False ALIAS_POST_TABLESAMPLE = False
# Determines whether or not unquoted identifiers are resolved as uppercase # Specifies the strategy according to which identifiers should be normalized.
# When set to None, it means that the dialect treats all identifiers as case-insensitive NORMALIZATION_STRATEGY = NormalizationStrategy.LOWERCASE
RESOLVES_IDENTIFIERS_AS_UPPERCASE: t.Optional[bool] = False
# Determines whether or not an unquoted identifier can start with a digit # Determines whether or not an unquoted identifier can start with a digit
IDENTIFIERS_CAN_START_WITH_DIGIT = False IDENTIFIERS_CAN_START_WITH_DIGIT = False
@ -177,6 +168,18 @@ class Dialect(metaclass=_Dialect):
# Options are: "nulls_are_small", "nulls_are_large", "nulls_are_last" # Options are: "nulls_are_small", "nulls_are_large", "nulls_are_last"
NULL_ORDERING = "nulls_are_small" NULL_ORDERING = "nulls_are_small"
# Whether the behavior of a / b depends on the types of a and b.
# False means a / b is always float division.
# True means a / b is integer division if both a and b are integers.
TYPED_DIVISION = False
# False means 1 / 0 throws an error.
# True means 1 / 0 returns null.
SAFE_DIVISION = False
# A NULL arg in CONCAT yields NULL by default, but in some dialects it yields an empty string
CONCAT_COALESCE = False
DATE_FORMAT = "'%Y-%m-%d'" DATE_FORMAT = "'%Y-%m-%d'"
DATEINT_FORMAT = "'%Y%m%d'" DATEINT_FORMAT = "'%Y%m%d'"
TIME_FORMAT = "'%Y-%m-%d %H:%M:%S'" TIME_FORMAT = "'%Y-%m-%d %H:%M:%S'"
@ -197,7 +200,8 @@ class Dialect(metaclass=_Dialect):
# Such columns may be excluded from SELECT * queries, for example # Such columns may be excluded from SELECT * queries, for example
PSEUDOCOLUMNS: t.Set[str] = set() PSEUDOCOLUMNS: t.Set[str] = set()
# Autofilled # --- Autofilled ---
tokenizer_class = Tokenizer tokenizer_class = Tokenizer
parser_class = Parser parser_class = Parser
generator_class = Generator generator_class = Generator
@ -211,26 +215,61 @@ class Dialect(metaclass=_Dialect):
INVERSE_ESCAPE_SEQUENCES: t.Dict[str, str] = {} INVERSE_ESCAPE_SEQUENCES: t.Dict[str, str] = {}
def __eq__(self, other: t.Any) -> bool: # Delimiters for quotes, identifiers and the corresponding escape characters
return type(self) == other QUOTE_START = "'"
QUOTE_END = "'"
IDENTIFIER_START = '"'
IDENTIFIER_END = '"'
def __hash__(self) -> int: # Delimiters for bit, hex and byte literals
return hash(type(self)) BIT_START: t.Optional[str] = None
BIT_END: t.Optional[str] = None
HEX_START: t.Optional[str] = None
HEX_END: t.Optional[str] = None
BYTE_START: t.Optional[str] = None
BYTE_END: t.Optional[str] = None
@classmethod @classmethod
def get_or_raise(cls, dialect: DialectType) -> t.Type[Dialect]: def get_or_raise(cls, dialect: DialectType) -> Dialect:
"""
Look up a dialect in the global dialect registry and return it if it exists.
Args:
dialect: The target dialect. If this is a string, it can be optionally followed by
additional key-value pairs that are separated by commas and are used to specify
dialect settings, such as whether the dialect's identifiers are case-sensitive.
Example:
>>> dialect = dialect_class = get_or_raise("duckdb")
>>> dialect = get_or_raise("mysql, normalization_strategy = case_sensitive")
Returns:
The corresponding Dialect instance.
"""
if not dialect: if not dialect:
return cls return cls()
if isinstance(dialect, _Dialect): if isinstance(dialect, _Dialect):
return dialect return dialect()
if isinstance(dialect, Dialect): if isinstance(dialect, Dialect):
return dialect.__class__ return dialect
if isinstance(dialect, str):
try:
dialect_name, *kv_pairs = dialect.split(",")
kwargs = {k.strip(): v.strip() for k, v in (kv.split("=") for kv in kv_pairs)}
except ValueError:
raise ValueError(
f"Invalid dialect format: '{dialect}'. "
"Please use the correct format: 'dialect [, k1 = v2 [, ...]]'."
)
result = cls.get(dialect) result = cls.get(dialect_name.strip())
if not result: if not result:
raise ValueError(f"Unknown dialect '{dialect}'") raise ValueError(f"Unknown dialect '{dialect_name}'.")
return result return result(**kwargs)
raise ValueError(f"Invalid dialect type for '{dialect}': '{type(dialect)}'.")
@classmethod @classmethod
def format_time( def format_time(
@ -247,36 +286,71 @@ class Dialect(metaclass=_Dialect):
return expression return expression
@classmethod def __init__(self, **kwargs) -> None:
def normalize_identifier(cls, expression: E) -> E: normalization_strategy = kwargs.get("normalization_strategy")
if normalization_strategy is None:
self.normalization_strategy = self.NORMALIZATION_STRATEGY
else:
self.normalization_strategy = NormalizationStrategy(normalization_strategy.upper())
def __eq__(self, other: t.Any) -> bool:
# Does not currently take dialect state into account
return type(self) == other
def __hash__(self) -> int:
# Does not currently take dialect state into account
return hash(type(self))
def normalize_identifier(self, expression: E) -> E:
""" """
Normalizes an unquoted identifier to either lower or upper case, thus essentially Transforms an identifier in a way that resembles how it'd be resolved by this dialect.
making it case-insensitive. If a dialect treats all identifiers as case-insensitive,
they will be normalized to lowercase regardless of being quoted or not. For example, an identifier like FoO would be resolved as foo in Postgres, because it
lowercases all unquoted identifiers. On the other hand, Snowflake uppercases them, so
it would resolve it as FOO. If it was quoted, it'd need to be treated as case-sensitive,
and so any normalization would be prohibited in order to avoid "breaking" the identifier.
There are also dialects like Spark, which are case-insensitive even when quotes are
present, and dialects like MySQL, whose resolution rules match those employed by the
underlying operating system, for example they may always be case-sensitive in Linux.
Finally, the normalization behavior of some engines can even be controlled through flags,
like in Redshift's case, where users can explicitly set enable_case_sensitive_identifier.
SQLGlot aims to understand and handle all of these different behaviors gracefully, so
that it can analyze queries in the optimizer and successfully capture their semantics.
""" """
if isinstance(expression, exp.Identifier) and ( if (
not expression.quoted or cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE is None isinstance(expression, exp.Identifier)
and not self.normalization_strategy is NormalizationStrategy.CASE_SENSITIVE
and (
not expression.quoted
or self.normalization_strategy is NormalizationStrategy.CASE_INSENSITIVE
)
): ):
expression.set( expression.set(
"this", "this",
expression.this.upper() expression.this.upper()
if cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE if self.normalization_strategy is NormalizationStrategy.UPPERCASE
else expression.this.lower(), else expression.this.lower(),
) )
return expression return expression
@classmethod def case_sensitive(self, text: str) -> bool:
def case_sensitive(cls, text: str) -> bool:
"""Checks if text contains any case sensitive characters, based on the dialect's rules.""" """Checks if text contains any case sensitive characters, based on the dialect's rules."""
if cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE is None: if self.normalization_strategy is NormalizationStrategy.CASE_INSENSITIVE:
return False return False
unsafe = str.islower if cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE else str.isupper unsafe = (
str.islower
if self.normalization_strategy is NormalizationStrategy.UPPERCASE
else str.isupper
)
return any(unsafe(char) for char in text) return any(unsafe(char) for char in text)
@classmethod def can_identify(self, text: str, identify: str | bool = "safe") -> bool:
def can_identify(cls, text: str, identify: str | bool = "safe") -> bool:
"""Checks if text can be identified given an identify option. """Checks if text can be identified given an identify option.
Args: Args:
@ -292,17 +366,16 @@ class Dialect(metaclass=_Dialect):
return True return True
if identify == "safe": if identify == "safe":
return not cls.case_sensitive(text) return not self.case_sensitive(text)
return False return False
@classmethod def quote_identifier(self, expression: E, identify: bool = True) -> E:
def quote_identifier(cls, expression: E, identify: bool = True) -> E:
if isinstance(expression, exp.Identifier): if isinstance(expression, exp.Identifier):
name = expression.this name = expression.this
expression.set( expression.set(
"quoted", "quoted",
identify or cls.case_sensitive(name) or not exp.SAFE_IDENTIFIER_RE.match(name), identify or self.case_sensitive(name) or not exp.SAFE_IDENTIFIER_RE.match(name),
) )
return expression return expression
@ -330,14 +403,14 @@ class Dialect(metaclass=_Dialect):
@property @property
def tokenizer(self) -> Tokenizer: def tokenizer(self) -> Tokenizer:
if not hasattr(self, "_tokenizer"): if not hasattr(self, "_tokenizer"):
self._tokenizer = self.tokenizer_class() self._tokenizer = self.tokenizer_class(dialect=self)
return self._tokenizer return self._tokenizer
def parser(self, **opts) -> Parser: def parser(self, **opts) -> Parser:
return self.parser_class(**opts) return self.parser_class(dialect=self, **opts)
def generator(self, **opts) -> Generator: def generator(self, **opts) -> Generator:
return self.generator_class(**opts) return self.generator_class(dialect=self, **opts)
DialectType = t.Union[str, Dialect, t.Type[Dialect], None] DialectType = t.Union[str, Dialect, t.Type[Dialect], None]
@ -713,7 +786,7 @@ def ts_or_ds_to_date_sql(dialect: str) -> t.Callable:
return _ts_or_ds_to_date_sql return _ts_or_ds_to_date_sql
def concat_to_dpipe_sql(self: Generator, expression: exp.Concat | exp.SafeConcat) -> str: def concat_to_dpipe_sql(self: Generator, expression: exp.Concat) -> str:
return self.sql(reduce(lambda x, y: exp.DPipe(this=x, expression=y), expression.expressions)) return self.sql(reduce(lambda x, y: exp.DPipe(this=x, expression=y), expression.expressions))
@ -821,3 +894,28 @@ def arg_max_or_min_no_count(name: str) -> t.Callable[[Generator, exp.ArgMax | ex
return self.func(name, expression.this, expression.expression) return self.func(name, expression.this, expression.expression)
return _arg_max_or_min_sql return _arg_max_or_min_sql
def ts_or_ds_add_cast(expression: exp.TsOrDsAdd) -> exp.TsOrDsAdd:
this = expression.this.copy()
return_type = expression.return_type
if return_type.is_type(exp.DataType.Type.DATE):
# If we need to cast to a DATE, we cast to TIMESTAMP first to make sure we
# can truncate timestamp strings, because some dialects can't cast them to DATE
this = exp.cast(this, exp.DataType.Type.TIMESTAMP)
expression.this.replace(exp.cast(this, return_type))
return expression
def date_delta_sql(name: str, cast: bool = False) -> t.Callable[[Generator, DATE_ADD_OR_DIFF], str]:
def _delta_sql(self: Generator, expression: DATE_ADD_OR_DIFF) -> str:
if cast and isinstance(expression, exp.TsOrDsAdd):
expression = ts_or_ds_add_cast(expression)
return self.func(
name, exp.var(expression.text("unit") or "day"), expression.expression, expression.this
)
return _delta_sql

3
sqlglot/dialects/doris.py
View file

@ -19,6 +19,7 @@ class Doris(MySQL):
class Parser(MySQL.Parser): class Parser(MySQL.Parser):
FUNCTIONS = { FUNCTIONS = {
**MySQL.Parser.FUNCTIONS, **MySQL.Parser.FUNCTIONS,
"COLLECT_SET": exp.ArrayUniqueAgg.from_arg_list,
"DATE_TRUNC": parse_timestamp_trunc, "DATE_TRUNC": parse_timestamp_trunc,
"REGEXP": exp.RegexpLike.from_arg_list, "REGEXP": exp.RegexpLike.from_arg_list,
} }
@ -47,7 +48,7 @@ class Doris(MySQL):
exp.JSONExtract: arrow_json_extract_sql, exp.JSONExtract: arrow_json_extract_sql,
exp.RegexpLike: rename_func("REGEXP"), exp.RegexpLike: rename_func("REGEXP"),
exp.RegexpSplit: rename_func("SPLIT_BY_STRING"), exp.RegexpSplit: rename_func("SPLIT_BY_STRING"),
exp.SetAgg: rename_func("COLLECT_SET"), exp.ArrayUniqueAgg: rename_func("COLLECT_SET"),
exp.StrToUnix: lambda self, e: f"UNIX_TIMESTAMP({self.sql(e, 'this')}, {self.format_time(e)})", exp.StrToUnix: lambda self, e: f"UNIX_TIMESTAMP({self.sql(e, 'this')}, {self.format_time(e)})",
exp.Split: rename_func("SPLIT_BY_STRING"), exp.Split: rename_func("SPLIT_BY_STRING"),
exp.TimeStrToDate: rename_func("TO_DATE"), exp.TimeStrToDate: rename_func("TO_DATE"),

3
sqlglot/dialects/drill.py
View file

@ -43,6 +43,8 @@ class Drill(Dialect):
TIME_FORMAT = "'yyyy-MM-dd HH:mm:ss'" TIME_FORMAT = "'yyyy-MM-dd HH:mm:ss'"
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
TYPED_DIVISION = True
CONCAT_COALESCE = True
TIME_MAPPING = { TIME_MAPPING = {
"y": "%Y", "y": "%Y",
@ -83,7 +85,6 @@ class Drill(Dialect):
class Parser(parser.Parser): class Parser(parser.Parser):
STRICT_CAST = False STRICT_CAST = False
CONCAT_NULL_OUTPUTS_STRING = True
FUNCTIONS = { FUNCTIONS = {
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,

71
sqlglot/dialects/duckdb.py
View file

@ -2,9 +2,10 @@ from __future__ import annotations
import typing as t import typing as t
from sqlglot import exp, generator, parser, tokens from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
approx_count_distinct_sql, approx_count_distinct_sql,
arg_max_or_min_no_count, arg_max_or_min_no_count,
arrow_json_extract_scalar_sql, arrow_json_extract_scalar_sql,
@ -36,7 +37,8 @@ from sqlglot.tokens import TokenType
def _ts_or_ds_add_sql(self: DuckDB.Generator, expression: exp.TsOrDsAdd) -> str: def _ts_or_ds_add_sql(self: DuckDB.Generator, expression: exp.TsOrDsAdd) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = self.sql(expression, "unit").strip("'") or "DAY" unit = self.sql(expression, "unit").strip("'") or "DAY"
return f"CAST({this} AS DATE) + {self.sql(exp.Interval(this=expression.expression, unit=unit))}" interval = self.sql(exp.Interval(this=expression.expression, unit=unit))
return f"CAST({this} AS {self.sql(expression.return_type)}) + {interval}"
def _date_delta_sql(self: DuckDB.Generator, expression: exp.DateAdd | exp.DateSub) -> str: def _date_delta_sql(self: DuckDB.Generator, expression: exp.DateAdd | exp.DateSub) -> str:
@ -84,7 +86,8 @@ def _parse_date_diff(args: t.List) -> exp.Expression:
def _struct_sql(self: DuckDB.Generator, expression: exp.Struct) -> str: def _struct_sql(self: DuckDB.Generator, expression: exp.Struct) -> str:
args = [ args = [
f"'{e.name or e.this.name}': {self.sql(e, 'expression')}" for e in expression.expressions f"'{e.name or e.this.name}': {self.sql(e.expressions[0]) if isinstance(e, exp.Bracket) else self.sql(e, 'expression')}"
for e in expression.expressions
] ]
return f"{{{', '.join(args)}}}" return f"{{{', '.join(args)}}}"
@ -105,17 +108,35 @@ def _json_format_sql(self: DuckDB.Generator, expression: exp.JSONFormat) -> str:
return f"CAST({sql} AS TEXT)" return f"CAST({sql} AS TEXT)"
def _unix_to_time_sql(self: DuckDB.Generator, expression: exp.UnixToTime) -> str:
scale = expression.args.get("scale")
timestamp = self.sql(expression, "this")
if scale in (None, exp.UnixToTime.SECONDS):
return f"TO_TIMESTAMP({timestamp})"
if scale == exp.UnixToTime.MILLIS:
return f"EPOCH_MS({timestamp})"
if scale == exp.UnixToTime.MICROS:
return f"MAKE_TIMESTAMP({timestamp})"
if scale == exp.UnixToTime.NANOS:
return f"TO_TIMESTAMP({timestamp} / 1000000000)"
self.unsupported(f"Unsupported scale for timestamp: {scale}.")
return ""
class DuckDB(Dialect): class DuckDB(Dialect):
NULL_ORDERING = "nulls_are_last" NULL_ORDERING = "nulls_are_last"
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
SAFE_DIVISION = True
INDEX_OFFSET = 1
CONCAT_COALESCE = True
# https://duckdb.org/docs/sql/introduction.html#creating-a-new-table # https://duckdb.org/docs/sql/introduction.html#creating-a-new-table
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
KEYWORDS = { KEYWORDS = {
**tokens.Tokenizer.KEYWORDS, **tokens.Tokenizer.KEYWORDS,
":=": TokenType.EQ,
"//": TokenType.DIV, "//": TokenType.DIV,
"ATTACH": TokenType.COMMAND, "ATTACH": TokenType.COMMAND,
"BINARY": TokenType.VARBINARY, "BINARY": TokenType.VARBINARY,
@ -124,8 +145,6 @@ class DuckDB(Dialect):
"CHAR": TokenType.TEXT, "CHAR": TokenType.TEXT,
"CHARACTER VARYING": TokenType.TEXT, "CHARACTER VARYING": TokenType.TEXT,
"EXCLUDE": TokenType.EXCEPT, "EXCLUDE": TokenType.EXCEPT,
"HUGEINT": TokenType.INT128,
"INT1": TokenType.TINYINT,
"LOGICAL": TokenType.BOOLEAN, "LOGICAL": TokenType.BOOLEAN,
"PIVOT_WIDER": TokenType.PIVOT, "PIVOT_WIDER": TokenType.PIVOT,
"SIGNED": TokenType.INT, "SIGNED": TokenType.INT,
@ -141,8 +160,6 @@ class DuckDB(Dialect):
} }
class Parser(parser.Parser): class Parser(parser.Parser):
CONCAT_NULL_OUTPUTS_STRING = True
BITWISE = { BITWISE = {
**parser.Parser.BITWISE, **parser.Parser.BITWISE,
TokenType.TILDA: exp.RegexpLike, TokenType.TILDA: exp.RegexpLike,
@ -150,6 +167,7 @@ class DuckDB(Dialect):
FUNCTIONS = { FUNCTIONS = {
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,
"ARRAY_HAS": exp.ArrayContains.from_arg_list,
"ARRAY_LENGTH": exp.ArraySize.from_arg_list, "ARRAY_LENGTH": exp.ArraySize.from_arg_list,
"ARRAY_SORT": exp.SortArray.from_arg_list, "ARRAY_SORT": exp.SortArray.from_arg_list,
"ARRAY_REVERSE_SORT": _sort_array_reverse, "ARRAY_REVERSE_SORT": _sort_array_reverse,
@ -157,13 +175,23 @@ class DuckDB(Dialect):
"DATE_DIFF": _parse_date_diff, "DATE_DIFF": _parse_date_diff,
"DATE_TRUNC": date_trunc_to_time, "DATE_TRUNC": date_trunc_to_time,
"DATETRUNC": date_trunc_to_time, "DATETRUNC": date_trunc_to_time,
"DECODE": lambda args: exp.Decode(
this=seq_get(args, 0), charset=exp.Literal.string("utf-8")
),
"ENCODE": lambda args: exp.Encode(
this=seq_get(args, 0), charset=exp.Literal.string("utf-8")
),
"EPOCH": exp.TimeToUnix.from_arg_list, "EPOCH": exp.TimeToUnix.from_arg_list,
"EPOCH_MS": lambda args: exp.UnixToTime( "EPOCH_MS": lambda args: exp.UnixToTime(
this=exp.Div(this=seq_get(args, 0), expression=exp.Literal.number(1000)) this=seq_get(args, 0), scale=exp.UnixToTime.MILLIS
), ),
"LIST_HAS": exp.ArrayContains.from_arg_list,
"LIST_REVERSE_SORT": _sort_array_reverse, "LIST_REVERSE_SORT": _sort_array_reverse,
"LIST_SORT": exp.SortArray.from_arg_list, "LIST_SORT": exp.SortArray.from_arg_list,
"LIST_VALUE": exp.Array.from_arg_list, "LIST_VALUE": exp.Array.from_arg_list,
"MAKE_TIMESTAMP": lambda args: exp.UnixToTime(
this=seq_get(args, 0), scale=exp.UnixToTime.MICROS
),
"MEDIAN": lambda args: exp.PercentileCont( "MEDIAN": lambda args: exp.PercentileCont(
this=seq_get(args, 0), expression=exp.Literal.number(0.5) this=seq_get(args, 0), expression=exp.Literal.number(0.5)
), ),
@ -192,15 +220,8 @@ class DuckDB(Dialect):
"XOR": binary_from_function(exp.BitwiseXor), "XOR": binary_from_function(exp.BitwiseXor),
} }
FUNCTION_PARSERS = { FUNCTION_PARSERS = parser.Parser.FUNCTION_PARSERS.copy()
**parser.Parser.FUNCTION_PARSERS, FUNCTION_PARSERS.pop("DECODE", None)
"DECODE": lambda self: self.expression(
exp.Decode, this=self._parse_conjunction(), charset=exp.Literal.string("utf-8")
),
"ENCODE": lambda self: self.expression(
exp.Encode, this=self._parse_conjunction(), charset=exp.Literal.string("utf-8")
),
}
TABLE_ALIAS_TOKENS = parser.Parser.TABLE_ALIAS_TOKENS - { TABLE_ALIAS_TOKENS = parser.Parser.TABLE_ALIAS_TOKENS - {
TokenType.SEMI, TokenType.SEMI,
@ -277,6 +298,7 @@ class DuckDB(Dialect):
exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False), exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False),
exp.Explode: rename_func("UNNEST"), exp.Explode: rename_func("UNNEST"),
exp.IntDiv: lambda self, e: self.binary(e, "//"), exp.IntDiv: lambda self, e: self.binary(e, "//"),
exp.IsInf: rename_func("ISINF"),
exp.IsNan: rename_func("ISNAN"), exp.IsNan: rename_func("ISNAN"),
exp.JSONExtract: arrow_json_extract_sql, exp.JSONExtract: arrow_json_extract_sql,
exp.JSONExtractScalar: arrow_json_extract_scalar_sql, exp.JSONExtractScalar: arrow_json_extract_scalar_sql,
@ -294,6 +316,9 @@ class DuckDB(Dialect):
exp.ParseJSON: rename_func("JSON"), exp.ParseJSON: rename_func("JSON"),
exp.PercentileCont: rename_func("QUANTILE_CONT"), exp.PercentileCont: rename_func("QUANTILE_CONT"),
exp.PercentileDisc: rename_func("QUANTILE_DISC"), exp.PercentileDisc: rename_func("QUANTILE_DISC"),
# DuckDB doesn't allow qualified columns inside of PIVOT expressions.
# See: https://github.com/duckdb/duckdb/blob/671faf92411182f81dce42ac43de8bfb05d9909e/src/planner/binder/tableref/bind_pivot.cpp#L61-L62
exp.Pivot: transforms.preprocess([transforms.unqualify_columns]),
exp.Properties: no_properties_sql, exp.Properties: no_properties_sql,
exp.RegexpExtract: regexp_extract_sql, exp.RegexpExtract: regexp_extract_sql,
exp.RegexpReplace: lambda self, e: self.func( exp.RegexpReplace: lambda self, e: self.func(
@ -322,9 +347,15 @@ class DuckDB(Dialect):
exp.TimeToUnix: rename_func("EPOCH"), exp.TimeToUnix: rename_func("EPOCH"),
exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS TEXT), '-', ''), 1, 8) AS INT)", exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS TEXT), '-', ''), 1, 8) AS INT)",
exp.TsOrDsAdd: _ts_or_ds_add_sql, exp.TsOrDsAdd: _ts_or_ds_add_sql,
exp.TsOrDsDiff: lambda self, e: self.func(
"DATE_DIFF",
f"'{e.args.get('unit') or 'day'}'",
exp.cast(e.expression, "TIMESTAMP"),
exp.cast(e.this, "TIMESTAMP"),
),
exp.TsOrDsToDate: ts_or_ds_to_date_sql("duckdb"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("duckdb"),
exp.UnixToStr: lambda self, e: f"STRFTIME(TO_TIMESTAMP({self.sql(e, 'this')}), {self.format_time(e)})", exp.UnixToStr: lambda self, e: f"STRFTIME(TO_TIMESTAMP({self.sql(e, 'this')}), {self.format_time(e)})",
exp.UnixToTime: rename_func("TO_TIMESTAMP"), exp.UnixToTime: _unix_to_time_sql,
exp.UnixToTimeStr: lambda self, e: f"CAST(TO_TIMESTAMP({self.sql(e, 'this')}) AS TEXT)", exp.UnixToTimeStr: lambda self, e: f"CAST(TO_TIMESTAMP({self.sql(e, 'this')}) AS TEXT)",
exp.VariancePop: rename_func("VAR_POP"), exp.VariancePop: rename_func("VAR_POP"),
exp.WeekOfYear: rename_func("WEEKOFYEAR"), exp.WeekOfYear: rename_func("WEEKOFYEAR"),

56
sqlglot/dialects/hive.py
View file

@ -4,10 +4,13 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
DATE_ADD_OR_SUB,
Dialect, Dialect,
NormalizationStrategy,
approx_count_distinct_sql, approx_count_distinct_sql,
arg_max_or_min_no_count, arg_max_or_min_no_count,
create_with_partitions_sql, create_with_partitions_sql,
datestrtodate_sql,
format_time_lambda, format_time_lambda,
if_sql, if_sql,
is_parse_json, is_parse_json,
@ -76,7 +79,10 @@ def _create_sql(self, expression: exp.Create) -> str:
return create_with_partitions_sql(self, expression) return create_with_partitions_sql(self, expression)
def _add_date_sql(self: Hive.Generator, expression: exp.DateAdd | exp.DateSub) -> str: def _add_date_sql(self: Hive.Generator, expression: DATE_ADD_OR_SUB) -> str:
if isinstance(expression, exp.TsOrDsAdd) and not expression.unit:
return self.func("DATE_ADD", expression.this, expression.expression)
unit = expression.text("unit").upper() unit = expression.text("unit").upper()
func, multiplier = DATE_DELTA_INTERVAL.get(unit, ("DATE_ADD", 1)) func, multiplier = DATE_DELTA_INTERVAL.get(unit, ("DATE_ADD", 1))
@ -95,7 +101,7 @@ def _add_date_sql(self: Hive.Generator, expression: exp.DateAdd | exp.DateSub) -
return self.func(func, expression.this, modified_increment) return self.func(func, expression.this, modified_increment)
def _date_diff_sql(self: Hive.Generator, expression: exp.DateDiff) -> str: def _date_diff_sql(self: Hive.Generator, expression: exp.DateDiff | exp.TsOrDsDiff) -> str:
unit = expression.text("unit").upper() unit = expression.text("unit").upper()
factor = TIME_DIFF_FACTOR.get(unit) factor = TIME_DIFF_FACTOR.get(unit)
@ -111,25 +117,31 @@ def _date_diff_sql(self: Hive.Generator, expression: exp.DateDiff) -> str:
multiplier_sql = f" / {multiplier}" if multiplier > 1 else "" multiplier_sql = f" / {multiplier}" if multiplier > 1 else ""
diff_sql = f"{sql_func}({self.format_args(expression.this, expression.expression)})" diff_sql = f"{sql_func}({self.format_args(expression.this, expression.expression)})"
if months_between: if months_between or multiplier_sql:
# MONTHS_BETWEEN returns a float, so we need to truncate the fractional part # MONTHS_BETWEEN returns a float, so we need to truncate the fractional part.
diff_sql = f"CAST({diff_sql} AS INT)" # For the same reason, we want to truncate if there's a divisor present.
diff_sql = f"CAST({diff_sql}{multiplier_sql} AS INT)"
return f"{diff_sql}{multiplier_sql}" return diff_sql
def _json_format_sql(self: Hive.Generator, expression: exp.JSONFormat) -> str: def _json_format_sql(self: Hive.Generator, expression: exp.JSONFormat) -> str:
this = expression.this this = expression.this
if is_parse_json(this) and this.this.is_string:
if is_parse_json(this):
if this.this.is_string:
# Since FROM_JSON requires a nested type, we always wrap the json string with # Since FROM_JSON requires a nested type, we always wrap the json string with
# an array to ensure that "naked" strings like "'a'" will be handled correctly # an array to ensure that "naked" strings like "'a'" will be handled correctly
wrapped_json = exp.Literal.string(f"[{this.this.name}]") wrapped_json = exp.Literal.string(f"[{this.this.name}]")
from_json = self.func("FROM_JSON", wrapped_json, self.func("SCHEMA_OF_JSON", wrapped_json)) from_json = self.func(
"FROM_JSON", wrapped_json, self.func("SCHEMA_OF_JSON", wrapped_json)
)
to_json = self.func("TO_JSON", from_json) to_json = self.func("TO_JSON", from_json)
# This strips the [, ] delimiters of the dummy array printed by TO_JSON # This strips the [, ] delimiters of the dummy array printed by TO_JSON
return self.func("REGEXP_EXTRACT", to_json, "'^.(.*).$'", "1") return self.func("REGEXP_EXTRACT", to_json, "'^.(.*).$'", "1")
return self.sql(this)
return self.func("TO_JSON", this, expression.args.get("options")) return self.func("TO_JSON", this, expression.args.get("options"))
@ -175,6 +187,8 @@ def _to_date_sql(self: Hive.Generator, expression: exp.TsOrDsToDate) -> str:
time_format = self.format_time(expression) time_format = self.format_time(expression)
if time_format and time_format not in (Hive.TIME_FORMAT, Hive.DATE_FORMAT): if time_format and time_format not in (Hive.TIME_FORMAT, Hive.DATE_FORMAT):
return f"TO_DATE({this}, {time_format})" return f"TO_DATE({this}, {time_format})"
if isinstance(expression.this, exp.TsOrDsToDate):
return this
return f"TO_DATE({this})" return f"TO_DATE({this})"
@ -182,9 +196,10 @@ class Hive(Dialect):
ALIAS_POST_TABLESAMPLE = True ALIAS_POST_TABLESAMPLE = True
IDENTIFIERS_CAN_START_WITH_DIGIT = True IDENTIFIERS_CAN_START_WITH_DIGIT = True
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
SAFE_DIVISION = True
# https://spark.apache.org/docs/latest/sql-ref-identifier.html#description # https://spark.apache.org/docs/latest/sql-ref-identifier.html#description
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
TIME_MAPPING = { TIME_MAPPING = {
"y": "%Y", "y": "%Y",
@ -241,10 +256,10 @@ class Hive(Dialect):
"ADD JAR": TokenType.COMMAND, "ADD JAR": TokenType.COMMAND,
"ADD JARS": TokenType.COMMAND, "ADD JARS": TokenType.COMMAND,
"MSCK REPAIR": TokenType.COMMAND, "MSCK REPAIR": TokenType.COMMAND,
"REFRESH": TokenType.COMMAND, "REFRESH": TokenType.REFRESH,
"WITH SERDEPROPERTIES": TokenType.SERDE_PROPERTIES,
"TIMESTAMP AS OF": TokenType.TIMESTAMP_SNAPSHOT, "TIMESTAMP AS OF": TokenType.TIMESTAMP_SNAPSHOT,
"VERSION AS OF": TokenType.VERSION_SNAPSHOT, "VERSION AS OF": TokenType.VERSION_SNAPSHOT,
"WITH SERDEPROPERTIES": TokenType.SERDE_PROPERTIES,
} }
NUMERIC_LITERALS = { NUMERIC_LITERALS = {
@ -264,7 +279,7 @@ class Hive(Dialect):
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,
"BASE64": exp.ToBase64.from_arg_list, "BASE64": exp.ToBase64.from_arg_list,
"COLLECT_LIST": exp.ArrayAgg.from_arg_list, "COLLECT_LIST": exp.ArrayAgg.from_arg_list,
"COLLECT_SET": exp.SetAgg.from_arg_list, "COLLECT_SET": exp.ArrayUniqueAgg.from_arg_list,
"DATE_ADD": lambda args: exp.TsOrDsAdd( "DATE_ADD": lambda args: exp.TsOrDsAdd(
this=seq_get(args, 0), expression=seq_get(args, 1), unit=exp.Literal.string("DAY") this=seq_get(args, 0), expression=seq_get(args, 1), unit=exp.Literal.string("DAY")
), ),
@ -411,7 +426,13 @@ class Hive(Dialect):
INDEX_ON = "ON TABLE" INDEX_ON = "ON TABLE"
EXTRACT_ALLOWS_QUOTES = False EXTRACT_ALLOWS_QUOTES = False
NVL2_SUPPORTED = False NVL2_SUPPORTED = False
SUPPORTS_NESTED_CTES = False
EXPRESSIONS_WITHOUT_NESTED_CTES = {
exp.Insert,
exp.Select,
exp.Subquery,
exp.Union,
}
TYPE_MAPPING = { TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING, **generator.Generator.TYPE_MAPPING,
@ -445,7 +466,7 @@ class Hive(Dialect):
exp.With: no_recursive_cte_sql, exp.With: no_recursive_cte_sql,
exp.DateAdd: _add_date_sql, exp.DateAdd: _add_date_sql,
exp.DateDiff: _date_diff_sql, exp.DateDiff: _date_diff_sql,
exp.DateStrToDate: rename_func("TO_DATE"), exp.DateStrToDate: datestrtodate_sql,
exp.DateSub: _add_date_sql, exp.DateSub: _add_date_sql,
exp.DateToDi: lambda self, e: f"CAST(DATE_FORMAT({self.sql(e, 'this')}, {Hive.DATEINT_FORMAT}) AS INT)", exp.DateToDi: lambda self, e: f"CAST(DATE_FORMAT({self.sql(e, 'this')}, {Hive.DATEINT_FORMAT}) AS INT)",
exp.DiToDate: lambda self, e: f"TO_DATE(CAST({self.sql(e, 'this')} AS STRING), {Hive.DATEINT_FORMAT})", exp.DiToDate: lambda self, e: f"TO_DATE(CAST({self.sql(e, 'this')} AS STRING), {Hive.DATEINT_FORMAT})",
@ -477,7 +498,7 @@ class Hive(Dialect):
exp.Right: right_to_substring_sql, exp.Right: right_to_substring_sql,
exp.SafeDivide: no_safe_divide_sql, exp.SafeDivide: no_safe_divide_sql,
exp.SchemaCommentProperty: lambda self, e: self.naked_property(e), exp.SchemaCommentProperty: lambda self, e: self.naked_property(e),
exp.SetAgg: rename_func("COLLECT_SET"), exp.ArrayUniqueAgg: rename_func("COLLECT_SET"),
exp.Split: lambda self, e: f"SPLIT({self.sql(e, 'this')}, CONCAT('\\\\Q', {self.sql(e, 'expression')}))", exp.Split: lambda self, e: f"SPLIT({self.sql(e, 'this')}, CONCAT('\\\\Q', {self.sql(e, 'expression')}))",
exp.StrPosition: strposition_to_locate_sql, exp.StrPosition: strposition_to_locate_sql,
exp.StrToDate: _str_to_date_sql, exp.StrToDate: _str_to_date_sql,
@ -491,7 +512,8 @@ class Hive(Dialect):
exp.TimeToUnix: rename_func("UNIX_TIMESTAMP"), exp.TimeToUnix: rename_func("UNIX_TIMESTAMP"),
exp.ToBase64: rename_func("BASE64"), exp.ToBase64: rename_func("BASE64"),
exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS STRING), '-', ''), 1, 8) AS INT)", exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS STRING), '-', ''), 1, 8) AS INT)",
exp.TsOrDsAdd: lambda self, e: f"DATE_ADD({self.sql(e, 'this')}, {self.sql(e, 'expression')})", exp.TsOrDsAdd: _add_date_sql,
exp.TsOrDsDiff: _date_diff_sql,
exp.TsOrDsToDate: _to_date_sql, exp.TsOrDsToDate: _to_date_sql,
exp.TryCast: no_trycast_sql, exp.TryCast: no_trycast_sql,
exp.UnixToStr: lambda self, e: self.func( exp.UnixToStr: lambda self, e: self.func(
@ -571,6 +593,8 @@ class Hive(Dialect):
and not expression.expressions and not expression.expressions
): ):
expression = exp.DataType.build("text") expression = exp.DataType.build("text")
elif expression.is_type(exp.DataType.Type.TEXT) and expression.expressions:
expression.set("this", exp.DataType.Type.VARCHAR)
elif expression.this in exp.DataType.TEMPORAL_TYPES: elif expression.this in exp.DataType.TEMPORAL_TYPES:
expression = exp.DataType.build(expression.this) expression = exp.DataType.build(expression.this)
elif expression.is_type("float"): elif expression.is_type("float"):

29
sqlglot/dialects/mysql.py
View file

@ -5,6 +5,7 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
arrow_json_extract_scalar_sql, arrow_json_extract_scalar_sql,
date_add_interval_sql, date_add_interval_sql,
datestrtodate_sql, datestrtodate_sql,
@ -150,10 +151,18 @@ class MySQL(Dialect):
# https://dev.mysql.com/doc/refman/8.0/en/identifiers.html # https://dev.mysql.com/doc/refman/8.0/en/identifiers.html
IDENTIFIERS_CAN_START_WITH_DIGIT = True IDENTIFIERS_CAN_START_WITH_DIGIT = True
# We default to treating all identifiers as case-sensitive, since it matches MySQL's
# behavior on Linux systems. For MacOS and Windows systems, one can override this
# setting by specifying `dialect="mysql, normalization_strategy = lowercase"`.
#
# See also https://dev.mysql.com/doc/refman/8.2/en/identifier-case-sensitivity.html
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE
TIME_FORMAT = "'%Y-%m-%d %T'" TIME_FORMAT = "'%Y-%m-%d %T'"
DPIPE_IS_STRING_CONCAT = False DPIPE_IS_STRING_CONCAT = False
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
SAFE_DIVISION = True
# https://prestodb.io/docs/current/functions/datetime.html#mysql-date-functions # https://prestodb.io/docs/current/functions/datetime.html#mysql-date-functions
TIME_MAPPING = { TIME_MAPPING = {
@ -264,11 +273,6 @@ class MySQL(Dialect):
TokenType.DPIPE: exp.Or, TokenType.DPIPE: exp.Or,
} }
# MySQL uses || as a synonym to the logical OR operator
# https://dev.mysql.com/doc/refman/8.0/en/logical-operators.html#operator_or
BITWISE = parser.Parser.BITWISE.copy()
BITWISE.pop(TokenType.DPIPE)
TABLE_ALIAS_TOKENS = ( TABLE_ALIAS_TOKENS = (
parser.Parser.TABLE_ALIAS_TOKENS - parser.Parser.TABLE_INDEX_HINT_TOKENS parser.Parser.TABLE_ALIAS_TOKENS - parser.Parser.TABLE_INDEX_HINT_TOKENS
) )
@ -451,7 +455,7 @@ class MySQL(Dialect):
self, kind: t.Optional[str] = None self, kind: t.Optional[str] = None
) -> exp.IndexColumnConstraint: ) -> exp.IndexColumnConstraint:
if kind: if kind:
self._match_texts({"INDEX", "KEY"}) self._match_texts(("INDEX", "KEY"))
this = self._parse_id_var(any_token=False) this = self._parse_id_var(any_token=False)
index_type = self._match(TokenType.USING) and self._advance_any() and self._prev.text index_type = self._match(TokenType.USING) and self._advance_any() and self._prev.text
@ -514,7 +518,7 @@ class MySQL(Dialect):
log = self._parse_string() if self._match_text_seq("IN") else None log = self._parse_string() if self._match_text_seq("IN") else None
if this in {"BINLOG EVENTS", "RELAYLOG EVENTS"}: if this in ("BINLOG EVENTS", "RELAYLOG EVENTS"):
position = self._parse_number() if self._match_text_seq("FROM") else None position = self._parse_number() if self._match_text_seq("FROM") else None
db = None db = None
else: else:
@ -671,6 +675,7 @@ class MySQL(Dialect):
exp.Trim: _trim_sql, exp.Trim: _trim_sql,
exp.TryCast: no_trycast_sql, exp.TryCast: no_trycast_sql,
exp.TsOrDsAdd: _date_add_sql("ADD"), exp.TsOrDsAdd: _date_add_sql("ADD"),
exp.TsOrDsDiff: lambda self, e: self.func("DATEDIFF", e.this, e.expression),
exp.TsOrDsToDate: _ts_or_ds_to_date_sql, exp.TsOrDsToDate: _ts_or_ds_to_date_sql,
exp.Week: _remove_ts_or_ds_to_date(), exp.Week: _remove_ts_or_ds_to_date(),
exp.WeekOfYear: _remove_ts_or_ds_to_date(rename_func("WEEKOFYEAR")), exp.WeekOfYear: _remove_ts_or_ds_to_date(rename_func("WEEKOFYEAR")),
@ -763,7 +768,7 @@ class MySQL(Dialect):
target = self.sql(expression, "target") target = self.sql(expression, "target")
target = f" {target}" if target else "" target = f" {target}" if target else ""
if expression.name in {"COLUMNS", "INDEX"}: if expression.name in ("COLUMNS", "INDEX"):
target = f" FROM{target}" target = f" FROM{target}"
elif expression.name == "GRANTS": elif expression.name == "GRANTS":
target = f" FOR{target}" target = f" FOR{target}"
@ -796,6 +801,14 @@ class MySQL(Dialect):
return f"SHOW{full}{global_}{this}{target}{types}{db}{query}{log}{position}{channel}{mutex_or_status}{like}{where}{offset}{limit}" return f"SHOW{full}{global_}{this}{target}{types}{db}{query}{log}{position}{channel}{mutex_or_status}{like}{where}{offset}{limit}"
def altercolumn_sql(self, expression: exp.AlterColumn) -> str:
dtype = self.sql(expression, "dtype")
if not dtype:
return super().altercolumn_sql(expression)
this = self.sql(expression, "this")
return f"MODIFY COLUMN {this} {dtype}"
def _prefixed_sql(self, prefix: str, expression: exp.Expression, arg: str) -> str: def _prefixed_sql(self, prefix: str, expression: exp.Expression, arg: str) -> str:
sql = self.sql(expression, arg) sql = self.sql(expression, arg)
return f" {prefix} {sql}" if sql else "" return f" {prefix} {sql}" if sql else ""

33
sqlglot/dialects/oracle.py
View file

@ -3,7 +3,14 @@ from __future__ import annotations
import typing as t import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import Dialect, no_ilike_sql, rename_func, trim_sql from sqlglot.dialects.dialect import (
Dialect,
NormalizationStrategy,
format_time_lambda,
no_ilike_sql,
rename_func,
trim_sql,
)
from sqlglot.helper import seq_get from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
@ -30,12 +37,25 @@ def _parse_xml_table(self: Oracle.Parser) -> exp.XMLTable:
return self.expression(exp.XMLTable, this=this, passing=passing, columns=columns, by_ref=by_ref) return self.expression(exp.XMLTable, this=this, passing=passing, columns=columns, by_ref=by_ref)
def to_char(args: t.List) -> exp.TimeToStr | exp.ToChar:
this = seq_get(args, 0)
if this and not this.type:
from sqlglot.optimizer.annotate_types import annotate_types
annotate_types(this)
if this.is_type(*exp.DataType.TEMPORAL_TYPES):
return format_time_lambda(exp.TimeToStr, "oracle", default=True)(args)
return exp.ToChar.from_arg_list(args)
class Oracle(Dialect): class Oracle(Dialect):
ALIAS_POST_TABLESAMPLE = True ALIAS_POST_TABLESAMPLE = True
LOCKING_READS_SUPPORTED = True LOCKING_READS_SUPPORTED = True
# See section 8: https://docs.oracle.com/cd/A97630_01/server.920/a96540/sql_elements9a.htm # See section 8: https://docs.oracle.com/cd/A97630_01/server.920/a96540/sql_elements9a.htm
RESOLVES_IDENTIFIERS_AS_UPPERCASE = True NORMALIZATION_STRATEGY = NormalizationStrategy.UPPERCASE
# https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212 # https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212
# https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes # https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
@ -64,11 +84,13 @@ class Oracle(Dialect):
} }
class Parser(parser.Parser): class Parser(parser.Parser):
ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = False
WINDOW_BEFORE_PAREN_TOKENS = {TokenType.OVER, TokenType.KEEP} WINDOW_BEFORE_PAREN_TOKENS = {TokenType.OVER, TokenType.KEEP}
FUNCTIONS = { FUNCTIONS = {
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,
"SQUARE": lambda args: exp.Pow(this=seq_get(args, 0), expression=exp.Literal.number(2)), "SQUARE": lambda args: exp.Pow(this=seq_get(args, 0), expression=exp.Literal.number(2)),
"TO_CHAR": to_char,
} }
FUNCTION_PARSERS: t.Dict[str, t.Callable] = { FUNCTION_PARSERS: t.Dict[str, t.Callable] = {
@ -130,6 +152,7 @@ class Oracle(Dialect):
TABLE_HINTS = False TABLE_HINTS = False
COLUMN_JOIN_MARKS_SUPPORTED = True COLUMN_JOIN_MARKS_SUPPORTED = True
DATA_TYPE_SPECIFIERS_ALLOWED = True DATA_TYPE_SPECIFIERS_ALLOWED = True
ALTER_TABLE_INCLUDE_COLUMN_KEYWORD = False
LIMIT_FETCH = "FETCH" LIMIT_FETCH = "FETCH"
@ -192,6 +215,12 @@ class Oracle(Dialect):
) )
return f"XMLTABLE({self.sep('')}{self.indent(this + passing + by_ref + columns)}{self.seg(')', sep='')}" return f"XMLTABLE({self.sep('')}{self.indent(this + passing + by_ref + columns)}{self.seg(')', sep='')}"
def add_column_sql(self, expression: exp.AlterTable) -> str:
actions = self.expressions(expression, key="actions", flat=True)
if len(expression.args.get("actions", [])) > 1:
return f"ADD ({actions})"
return f"ADD {actions}"
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
VAR_SINGLE_TOKENS = {"@", "$", "#"} VAR_SINGLE_TOKENS = {"@", "$", "#"}

61
sqlglot/dialects/postgres.py
View file

@ -4,6 +4,7 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
DATE_ADD_OR_SUB,
Dialect, Dialect,
any_value_to_max_sql, any_value_to_max_sql,
arrow_json_extract_scalar_sql, arrow_json_extract_scalar_sql,
@ -25,6 +26,7 @@ from sqlglot.dialects.dialect import (
timestamptrunc_sql, timestamptrunc_sql,
timestrtotime_sql, timestrtotime_sql,
trim_sql, trim_sql,
ts_or_ds_add_cast,
ts_or_ds_to_date_sql, ts_or_ds_to_date_sql,
) )
from sqlglot.helper import seq_get from sqlglot.helper import seq_get
@ -41,8 +43,11 @@ DATE_DIFF_FACTOR = {
} }
def _date_add_sql(kind: str) -> t.Callable[[Postgres.Generator, exp.DateAdd | exp.DateSub], str]: def _date_add_sql(kind: str) -> t.Callable[[Postgres.Generator, DATE_ADD_OR_SUB], str]:
def func(self: Postgres.Generator, expression: exp.DateAdd | exp.DateSub) -> str: def func(self: Postgres.Generator, expression: DATE_ADD_OR_SUB) -> str:
if isinstance(expression, exp.TsOrDsAdd):
expression = ts_or_ds_add_cast(expression)
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = expression.args.get("unit") unit = expression.args.get("unit")
@ -60,8 +65,8 @@ def _date_diff_sql(self: Postgres.Generator, expression: exp.DateDiff) -> str:
unit = expression.text("unit").upper() unit = expression.text("unit").upper()
factor = DATE_DIFF_FACTOR.get(unit) factor = DATE_DIFF_FACTOR.get(unit)
end = f"CAST({expression.this} AS TIMESTAMP)" end = f"CAST({self.sql(expression, 'this')} AS TIMESTAMP)"
start = f"CAST({expression.expression} AS TIMESTAMP)" start = f"CAST({self.sql(expression, 'expression')} AS TIMESTAMP)"
if factor is not None: if factor is not None:
return f"CAST(EXTRACT(epoch FROM {end} - {start}){factor} AS BIGINT)" return f"CAST(EXTRACT(epoch FROM {end} - {start}){factor} AS BIGINT)"
@ -69,7 +74,7 @@ def _date_diff_sql(self: Postgres.Generator, expression: exp.DateDiff) -> str:
age = f"AGE({end}, {start})" age = f"AGE({end}, {start})"
if unit == "WEEK": if unit == "WEEK":
unit = f"EXTRACT(year FROM {age}) * 48 + EXTRACT(month FROM {age}) * 4 + EXTRACT(day FROM {age}) / 7" unit = f"EXTRACT(days FROM ({end} - {start})) / 7"
elif unit == "MONTH": elif unit == "MONTH":
unit = f"EXTRACT(year FROM {age}) * 12 + EXTRACT(month FROM {age})" unit = f"EXTRACT(year FROM {age}) * 12 + EXTRACT(month FROM {age})"
elif unit == "QUARTER": elif unit == "QUARTER":
@ -183,13 +188,14 @@ def _to_timestamp(args: t.List) -> exp.Expression:
return format_time_lambda(exp.StrToTime, "postgres")(args) return format_time_lambda(exp.StrToTime, "postgres")(args)
def _remove_target_from_merge(expression: exp.Expression) -> exp.Expression: def _merge_sql(self: Postgres.Generator, expression: exp.Merge) -> str:
def _remove_target_from_merge(expression: exp.Expression) -> exp.Expression:
"""Remove table refs from columns in when statements.""" """Remove table refs from columns in when statements."""
if isinstance(expression, exp.Merge): if isinstance(expression, exp.Merge):
alias = expression.this.args.get("alias") alias = expression.this.args.get("alias")
normalize = ( normalize = (
lambda identifier: Postgres.normalize_identifier(identifier).name lambda identifier: self.dialect.normalize_identifier(identifier).name
if identifier if identifier
else None else None
) )
@ -209,11 +215,16 @@ def _remove_target_from_merge(expression: exp.Expression) -> exp.Expression:
return expression return expression
return transforms.preprocess([_remove_target_from_merge])(self, expression)
class Postgres(Dialect): class Postgres(Dialect):
INDEX_OFFSET = 1 INDEX_OFFSET = 1
TYPED_DIVISION = True
CONCAT_COALESCE = True
NULL_ORDERING = "nulls_are_large" NULL_ORDERING = "nulls_are_large"
TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'" TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'"
TIME_MAPPING = { TIME_MAPPING = {
"AM": "%p", "AM": "%p",
"PM": "%p", "PM": "%p",
@ -263,6 +274,7 @@ class Postgres(Dialect):
"BEGIN TRANSACTION": TokenType.BEGIN, "BEGIN TRANSACTION": TokenType.BEGIN,
"BIGSERIAL": TokenType.BIGSERIAL, "BIGSERIAL": TokenType.BIGSERIAL,
"CHARACTER VARYING": TokenType.VARCHAR, "CHARACTER VARYING": TokenType.VARCHAR,
"CONSTRAINT TRIGGER": TokenType.COMMAND,
"DECLARE": TokenType.COMMAND, "DECLARE": TokenType.COMMAND,
"DO": TokenType.COMMAND, "DO": TokenType.COMMAND,
"HSTORE": TokenType.HSTORE, "HSTORE": TokenType.HSTORE,
@ -277,6 +289,7 @@ class Postgres(Dialect):
"TEMP": TokenType.TEMPORARY, "TEMP": TokenType.TEMPORARY,
"CSTRING": TokenType.PSEUDO_TYPE, "CSTRING": TokenType.PSEUDO_TYPE,
"OID": TokenType.OBJECT_IDENTIFIER, "OID": TokenType.OBJECT_IDENTIFIER,
"OPERATOR": TokenType.OPERATOR,
"REGCLASS": TokenType.OBJECT_IDENTIFIER, "REGCLASS": TokenType.OBJECT_IDENTIFIER,
"REGCOLLATION": TokenType.OBJECT_IDENTIFIER, "REGCOLLATION": TokenType.OBJECT_IDENTIFIER,
"REGCONFIG": TokenType.OBJECT_IDENTIFIER, "REGCONFIG": TokenType.OBJECT_IDENTIFIER,
@ -298,8 +311,6 @@ class Postgres(Dialect):
VAR_SINGLE_TOKENS = {"$"} VAR_SINGLE_TOKENS = {"$"}
class Parser(parser.Parser): class Parser(parser.Parser):
CONCAT_NULL_OUTPUTS_STRING = True
FUNCTIONS = { FUNCTIONS = {
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,
"DATE_TRUNC": parse_timestamp_trunc, "DATE_TRUNC": parse_timestamp_trunc,
@ -326,12 +337,13 @@ class Postgres(Dialect):
RANGE_PARSERS = { RANGE_PARSERS = {
**parser.Parser.RANGE_PARSERS, **parser.Parser.RANGE_PARSERS,
TokenType.AT_GT: binary_range_parser(exp.ArrayContains),
TokenType.DAMP: binary_range_parser(exp.ArrayOverlaps), TokenType.DAMP: binary_range_parser(exp.ArrayOverlaps),
TokenType.DAT: lambda self, this: self.expression( TokenType.DAT: lambda self, this: self.expression(
exp.MatchAgainst, this=self._parse_bitwise(), expressions=[this] exp.MatchAgainst, this=self._parse_bitwise(), expressions=[this]
), ),
TokenType.AT_GT: binary_range_parser(exp.ArrayContains),
TokenType.LT_AT: binary_range_parser(exp.ArrayContained), TokenType.LT_AT: binary_range_parser(exp.ArrayContained),
TokenType.OPERATOR: lambda self, this: self._parse_operator(this),
} }
STATEMENT_PARSERS = { STATEMENT_PARSERS = {
@ -339,11 +351,28 @@ class Postgres(Dialect):
TokenType.END: lambda self: self._parse_commit_or_rollback(), TokenType.END: lambda self: self._parse_commit_or_rollback(),
} }
def _parse_factor(self) -> t.Optional[exp.Expression]: def _parse_operator(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
return self._parse_tokens(self._parse_exponent, self.FACTOR) while True:
if not self._match(TokenType.L_PAREN):
break
def _parse_exponent(self) -> t.Optional[exp.Expression]: op = ""
return self._parse_tokens(self._parse_unary, self.EXPONENT) while self._curr and not self._match(TokenType.R_PAREN):
op += self._curr.text
self._advance()
this = self.expression(
exp.Operator,
comments=self._prev_comments,
this=this,
operator=op,
expression=self._parse_bitwise(),
)
if not self._match(TokenType.OPERATOR):
break
return this
def _parse_date_part(self) -> exp.Expression: def _parse_date_part(self) -> exp.Expression:
part = self._parse_type() part = self._parse_type()
@ -405,7 +434,7 @@ class Postgres(Dialect):
exp.Max: max_or_greatest, exp.Max: max_or_greatest,
exp.MapFromEntries: no_map_from_entries_sql, exp.MapFromEntries: no_map_from_entries_sql,
exp.Min: min_or_least, exp.Min: min_or_least,
exp.Merge: transforms.preprocess([_remove_target_from_merge]), exp.Merge: _merge_sql,
exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}", exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}",
exp.PercentileCont: transforms.preprocess( exp.PercentileCont: transforms.preprocess(
[transforms.add_within_group_for_percentiles] [transforms.add_within_group_for_percentiles]
@ -434,6 +463,8 @@ class Postgres(Dialect):
exp.ToChar: lambda self, e: self.function_fallback_sql(e), exp.ToChar: lambda self, e: self.function_fallback_sql(e),
exp.Trim: trim_sql, exp.Trim: trim_sql,
exp.TryCast: no_trycast_sql, exp.TryCast: no_trycast_sql,
exp.TsOrDsAdd: _date_add_sql("+"),
exp.TsOrDsDiff: _date_diff_sql,
exp.TsOrDsToDate: ts_or_ds_to_date_sql("postgres"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("postgres"),
exp.UnixToTime: lambda self, e: f"TO_TIMESTAMP({self.sql(e, 'this')})", exp.UnixToTime: lambda self, e: f"TO_TIMESTAMP({self.sql(e, 'this')})",
exp.VariancePop: rename_func("VAR_POP"), exp.VariancePop: rename_func("VAR_POP"),

72
sqlglot/dialects/presto.py
View file

@ -5,9 +5,11 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
binary_from_function, binary_from_function,
bool_xor_sql, bool_xor_sql,
date_trunc_to_time, date_trunc_to_time,
datestrtodate_sql,
encode_decode_sql, encode_decode_sql,
format_time_lambda, format_time_lambda,
if_sql, if_sql,
@ -22,6 +24,7 @@ from sqlglot.dialects.dialect import (
struct_extract_sql, struct_extract_sql,
timestamptrunc_sql, timestamptrunc_sql,
timestrtotime_sql, timestrtotime_sql,
ts_or_ds_add_cast,
) )
from sqlglot.dialects.mysql import MySQL from sqlglot.dialects.mysql import MySQL
from sqlglot.helper import apply_index_offset, seq_get from sqlglot.helper import apply_index_offset, seq_get
@ -95,17 +98,16 @@ def _ts_or_ds_to_date_sql(self: Presto.Generator, expression: exp.TsOrDsToDate)
def _ts_or_ds_add_sql(self: Presto.Generator, expression: exp.TsOrDsAdd) -> str: def _ts_or_ds_add_sql(self: Presto.Generator, expression: exp.TsOrDsAdd) -> str:
this = expression.this expression = ts_or_ds_add_cast(expression)
unit = exp.Literal.string(expression.text("unit") or "day")
return self.func("DATE_ADD", unit, expression.expression, expression.this)
if not isinstance(this, exp.CurrentDate):
this = exp.cast(exp.cast(expression.this, "TIMESTAMP", copy=True), "DATE")
return self.func( def _ts_or_ds_diff_sql(self: Presto.Generator, expression: exp.TsOrDsDiff) -> str:
"DATE_ADD", this = exp.cast(expression.this, "TIMESTAMP")
exp.Literal.string(expression.text("unit") or "day"), expr = exp.cast(expression.expression, "TIMESTAMP")
expression.expression, unit = exp.Literal.string(expression.text("unit") or "day")
this, return self.func("DATE_DIFF", unit, expr, this)
)
def _approx_percentile(args: t.List) -> exp.Expression: def _approx_percentile(args: t.List) -> exp.Expression:
@ -136,11 +138,11 @@ def _from_unixtime(args: t.List) -> exp.Expression:
return exp.UnixToTime.from_arg_list(args) return exp.UnixToTime.from_arg_list(args)
def _parse_element_at(args: t.List) -> exp.SafeBracket: def _parse_element_at(args: t.List) -> exp.Bracket:
this = seq_get(args, 0) this = seq_get(args, 0)
index = seq_get(args, 1) index = seq_get(args, 1)
assert isinstance(this, exp.Expression) and isinstance(index, exp.Expression) assert isinstance(this, exp.Expression) and isinstance(index, exp.Expression)
return exp.SafeBracket(this=this, expressions=apply_index_offset(this, [index], -1)) return exp.Bracket(this=this, expressions=[index], offset=1, safe=True)
def _unnest_sequence(expression: exp.Expression) -> exp.Expression: def _unnest_sequence(expression: exp.Expression) -> exp.Expression:
@ -168,6 +170,22 @@ def _first_last_sql(self: Presto.Generator, expression: exp.First | exp.Last) ->
return rename_func("ARBITRARY")(self, expression) return rename_func("ARBITRARY")(self, expression)
def _unix_to_time_sql(self: Presto.Generator, expression: exp.UnixToTime) -> str:
scale = expression.args.get("scale")
timestamp = self.sql(expression, "this")
if scale in (None, exp.UnixToTime.SECONDS):
return rename_func("FROM_UNIXTIME")(self, expression)
if scale == exp.UnixToTime.MILLIS:
return f"FROM_UNIXTIME(CAST({timestamp} AS DOUBLE) / 1000)"
if scale == exp.UnixToTime.MICROS:
return f"FROM_UNIXTIME(CAST({timestamp} AS DOUBLE) / 1000000)"
if scale == exp.UnixToTime.NANOS:
return f"FROM_UNIXTIME(CAST({timestamp} AS DOUBLE) / 1000000000)"
self.unsupported(f"Unsupported scale for timestamp: {scale}.")
return ""
class Presto(Dialect): class Presto(Dialect):
INDEX_OFFSET = 1 INDEX_OFFSET = 1
NULL_ORDERING = "nulls_are_last" NULL_ORDERING = "nulls_are_last"
@ -175,11 +193,12 @@ class Presto(Dialect):
TIME_MAPPING = MySQL.TIME_MAPPING TIME_MAPPING = MySQL.TIME_MAPPING
STRICT_STRING_CONCAT = True STRICT_STRING_CONCAT = True
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
TYPED_DIVISION = True
# https://github.com/trinodb/trino/issues/17 # https://github.com/trinodb/trino/issues/17
# https://github.com/trinodb/trino/issues/12289 # https://github.com/trinodb/trino/issues/12289
# https://github.com/prestodb/presto/issues/2863 # https://github.com/prestodb/presto/issues/2863
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
KEYWORDS = { KEYWORDS = {
@ -229,6 +248,7 @@ class Presto(Dialect):
), ),
"ROW": exp.Struct.from_arg_list, "ROW": exp.Struct.from_arg_list,
"SEQUENCE": exp.GenerateSeries.from_arg_list, "SEQUENCE": exp.GenerateSeries.from_arg_list,
"SET_AGG": exp.ArrayUniqueAgg.from_arg_list,
"SPLIT_TO_MAP": exp.StrToMap.from_arg_list, "SPLIT_TO_MAP": exp.StrToMap.from_arg_list,
"STRPOS": lambda args: exp.StrPosition( "STRPOS": lambda args: exp.StrPosition(
this=seq_get(args, 0), substr=seq_get(args, 1), instance=seq_get(args, 2) this=seq_get(args, 0), substr=seq_get(args, 1), instance=seq_get(args, 2)
@ -253,6 +273,7 @@ class Presto(Dialect):
NVL2_SUPPORTED = False NVL2_SUPPORTED = False
STRUCT_DELIMITER = ("(", ")") STRUCT_DELIMITER = ("(", ")")
LIMIT_ONLY_LITERALS = True LIMIT_ONLY_LITERALS = True
SUPPORTS_SINGLE_ARG_CONCAT = False
PROPERTIES_LOCATION = { PROPERTIES_LOCATION = {
**generator.Generator.PROPERTIES_LOCATION, **generator.Generator.PROPERTIES_LOCATION,
@ -284,6 +305,7 @@ class Presto(Dialect):
exp.ArrayConcat: rename_func("CONCAT"), exp.ArrayConcat: rename_func("CONCAT"),
exp.ArrayContains: rename_func("CONTAINS"), exp.ArrayContains: rename_func("CONTAINS"),
exp.ArraySize: rename_func("CARDINALITY"), exp.ArraySize: rename_func("CARDINALITY"),
exp.ArrayUniqueAgg: rename_func("SET_AGG"),
exp.BitwiseAnd: lambda self, e: f"BITWISE_AND({self.sql(e, 'this')}, {self.sql(e, 'expression')})", exp.BitwiseAnd: lambda self, e: f"BITWISE_AND({self.sql(e, 'this')}, {self.sql(e, 'expression')})",
exp.BitwiseLeftShift: lambda self, e: f"BITWISE_ARITHMETIC_SHIFT_LEFT({self.sql(e, 'this')}, {self.sql(e, 'expression')})", exp.BitwiseLeftShift: lambda self, e: f"BITWISE_ARITHMETIC_SHIFT_LEFT({self.sql(e, 'this')}, {self.sql(e, 'expression')})",
exp.BitwiseNot: lambda self, e: f"BITWISE_NOT({self.sql(e, 'this')})", exp.BitwiseNot: lambda self, e: f"BITWISE_NOT({self.sql(e, 'this')})",
@ -298,7 +320,7 @@ class Presto(Dialect):
exp.DateDiff: lambda self, e: self.func( exp.DateDiff: lambda self, e: self.func(
"DATE_DIFF", exp.Literal.string(e.text("unit") or "day"), e.expression, e.this "DATE_DIFF", exp.Literal.string(e.text("unit") or "day"), e.expression, e.this
), ),
exp.DateStrToDate: lambda self, e: f"CAST(DATE_PARSE({self.sql(e, 'this')}, {Presto.DATE_FORMAT}) AS DATE)", exp.DateStrToDate: datestrtodate_sql,
exp.DateToDi: lambda self, e: f"CAST(DATE_FORMAT({self.sql(e, 'this')}, {Presto.DATEINT_FORMAT}) AS INT)", exp.DateToDi: lambda self, e: f"CAST(DATE_FORMAT({self.sql(e, 'this')}, {Presto.DATEINT_FORMAT}) AS INT)",
exp.DateSub: lambda self, e: self.func( exp.DateSub: lambda self, e: self.func(
"DATE_ADD", "DATE_ADD",
@ -330,9 +352,6 @@ class Presto(Dialect):
exp.Quantile: _quantile_sql, exp.Quantile: _quantile_sql,
exp.RegexpExtract: regexp_extract_sql, exp.RegexpExtract: regexp_extract_sql,
exp.Right: right_to_substring_sql, exp.Right: right_to_substring_sql,
exp.SafeBracket: lambda self, e: self.func(
"ELEMENT_AT", e.this, seq_get(apply_index_offset(e.this, e.expressions, 1), 0)
),
exp.SafeDivide: no_safe_divide_sql, exp.SafeDivide: no_safe_divide_sql,
exp.Schema: _schema_sql, exp.Schema: _schema_sql,
exp.Select: transforms.preprocess( exp.Select: transforms.preprocess(
@ -361,10 +380,11 @@ class Presto(Dialect):
exp.TryCast: transforms.preprocess([transforms.epoch_cast_to_ts]), exp.TryCast: transforms.preprocess([transforms.epoch_cast_to_ts]),
exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS VARCHAR), '-', ''), 1, 8) AS INT)", exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS VARCHAR), '-', ''), 1, 8) AS INT)",
exp.TsOrDsAdd: _ts_or_ds_add_sql, exp.TsOrDsAdd: _ts_or_ds_add_sql,
exp.TsOrDsDiff: _ts_or_ds_diff_sql,
exp.TsOrDsToDate: _ts_or_ds_to_date_sql, exp.TsOrDsToDate: _ts_or_ds_to_date_sql,
exp.Unhex: rename_func("FROM_HEX"), exp.Unhex: rename_func("FROM_HEX"),
exp.UnixToStr: lambda self, e: f"DATE_FORMAT(FROM_UNIXTIME({self.sql(e, 'this')}), {self.format_time(e)})", exp.UnixToStr: lambda self, e: f"DATE_FORMAT(FROM_UNIXTIME({self.sql(e, 'this')}), {self.format_time(e)})",
exp.UnixToTime: rename_func("FROM_UNIXTIME"), exp.UnixToTime: _unix_to_time_sql,
exp.UnixToTimeStr: lambda self, e: f"CAST(FROM_UNIXTIME({self.sql(e, 'this')}) AS VARCHAR)", exp.UnixToTimeStr: lambda self, e: f"CAST(FROM_UNIXTIME({self.sql(e, 'this')}) AS VARCHAR)",
exp.VariancePop: rename_func("VAR_POP"), exp.VariancePop: rename_func("VAR_POP"),
exp.With: transforms.preprocess([transforms.add_recursive_cte_column_names]), exp.With: transforms.preprocess([transforms.add_recursive_cte_column_names]),
@ -374,8 +394,24 @@ class Presto(Dialect):
exp.Xor: bool_xor_sql, exp.Xor: bool_xor_sql,
} }
def bracket_sql(self, expression: exp.Bracket) -> str:
if expression.args.get("safe"):
return self.func(
"ELEMENT_AT",
expression.this,
seq_get(
apply_index_offset(
expression.this,
expression.expressions,
1 - expression.args.get("offset", 0),
),
0,
),
)
return super().bracket_sql(expression)
def struct_sql(self, expression: exp.Struct) -> str: def struct_sql(self, expression: exp.Struct) -> str:
if any(isinstance(arg, (exp.EQ, exp.Slice)) for arg in expression.expressions): if any(isinstance(arg, self.KEY_VALUE_DEFINITONS) for arg in expression.expressions):
self.unsupported("Struct with key-value definitions is unsupported.") self.unsupported("Struct with key-value definitions is unsupported.")
return self.function_fallback_sql(expression) return self.function_fallback_sql(expression)

50
sqlglot/dialects/redshift.py
View file

@ -4,8 +4,10 @@ import typing as t
from sqlglot import exp, transforms from sqlglot import exp, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
NormalizationStrategy,
concat_to_dpipe_sql, concat_to_dpipe_sql,
concat_ws_to_dpipe_sql, concat_ws_to_dpipe_sql,
date_delta_sql,
generatedasidentitycolumnconstraint_sql, generatedasidentitycolumnconstraint_sql,
rename_func, rename_func,
ts_or_ds_to_date_sql, ts_or_ds_to_date_sql,
@ -14,30 +16,28 @@ from sqlglot.dialects.postgres import Postgres
from sqlglot.helper import seq_get from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
if t.TYPE_CHECKING:
from sqlglot._typing import E
def _json_sql(self: Redshift.Generator, expression: exp.JSONExtract | exp.JSONExtractScalar) -> str: def _json_sql(self: Redshift.Generator, expression: exp.JSONExtract | exp.JSONExtractScalar) -> str:
return f'{self.sql(expression, "this")}."{expression.expression.name}"' return f'{self.sql(expression, "this")}."{expression.expression.name}"'
def _parse_date_add(args: t.List) -> exp.DateAdd: def _parse_date_delta(expr_type: t.Type[E]) -> t.Callable[[t.List], E]:
return exp.DateAdd( def _parse_delta(args: t.List) -> E:
this=exp.TsOrDsToDate(this=seq_get(args, 2)), expr = expr_type(this=seq_get(args, 2), expression=seq_get(args, 1), unit=seq_get(args, 0))
expression=seq_get(args, 1), if expr_type is exp.TsOrDsAdd:
unit=seq_get(args, 0), expr.set("return_type", exp.DataType.build("TIMESTAMP"))
)
return expr
def _parse_datediff(args: t.List) -> exp.DateDiff: return _parse_delta
return exp.DateDiff(
this=exp.TsOrDsToDate(this=seq_get(args, 2)),
expression=exp.TsOrDsToDate(this=seq_get(args, 1)),
unit=seq_get(args, 0),
)
class Redshift(Postgres): class Redshift(Postgres):
# https://docs.aws.amazon.com/redshift/latest/dg/r_names.html # https://docs.aws.amazon.com/redshift/latest/dg/r_names.html
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
INDEX_OFFSET = 0 INDEX_OFFSET = 0
@ -52,15 +52,16 @@ class Redshift(Postgres):
class Parser(Postgres.Parser): class Parser(Postgres.Parser):
FUNCTIONS = { FUNCTIONS = {
**Postgres.Parser.FUNCTIONS, **Postgres.Parser.FUNCTIONS,
"ADD_MONTHS": lambda args: exp.DateAdd( "ADD_MONTHS": lambda args: exp.TsOrDsAdd(
this=exp.TsOrDsToDate(this=seq_get(args, 0)), this=seq_get(args, 0),
expression=seq_get(args, 1), expression=seq_get(args, 1),
unit=exp.var("month"), unit=exp.var("month"),
return_type=exp.DataType.build("TIMESTAMP"),
), ),
"DATEADD": _parse_date_add, "DATEADD": _parse_date_delta(exp.TsOrDsAdd),
"DATE_ADD": _parse_date_add, "DATE_ADD": _parse_date_delta(exp.TsOrDsAdd),
"DATEDIFF": _parse_datediff, "DATEDIFF": _parse_date_delta(exp.TsOrDsDiff),
"DATE_DIFF": _parse_datediff, "DATE_DIFF": _parse_date_delta(exp.TsOrDsDiff),
"LISTAGG": exp.GroupConcat.from_arg_list, "LISTAGG": exp.GroupConcat.from_arg_list,
"STRTOL": exp.FromBase.from_arg_list, "STRTOL": exp.FromBase.from_arg_list,
} }
@ -169,12 +170,8 @@ class Redshift(Postgres):
exp.ConcatWs: concat_ws_to_dpipe_sql, exp.ConcatWs: concat_ws_to_dpipe_sql,
exp.ApproxDistinct: lambda self, e: f"APPROXIMATE COUNT(DISTINCT {self.sql(e, 'this')})", exp.ApproxDistinct: lambda self, e: f"APPROXIMATE COUNT(DISTINCT {self.sql(e, 'this')})",
exp.CurrentTimestamp: lambda self, e: "SYSDATE", exp.CurrentTimestamp: lambda self, e: "SYSDATE",
exp.DateAdd: lambda self, e: self.func( exp.DateAdd: date_delta_sql("DATEADD"),
"DATEADD", exp.var(e.text("unit") or "day"), e.expression, e.this exp.DateDiff: date_delta_sql("DATEDIFF"),
),
exp.DateDiff: lambda self, e: self.func(
"DATEDIFF", exp.var(e.text("unit") or "day"), e.expression, e.this
),
exp.DistKeyProperty: lambda self, e: f"DISTKEY({e.name})", exp.DistKeyProperty: lambda self, e: f"DISTKEY({e.name})",
exp.DistStyleProperty: lambda self, e: self.naked_property(e), exp.DistStyleProperty: lambda self, e: self.naked_property(e),
exp.FromBase: rename_func("STRTOL"), exp.FromBase: rename_func("STRTOL"),
@ -183,11 +180,12 @@ class Redshift(Postgres):
exp.JSONExtractScalar: _json_sql, exp.JSONExtractScalar: _json_sql,
exp.GroupConcat: rename_func("LISTAGG"), exp.GroupConcat: rename_func("LISTAGG"),
exp.ParseJSON: rename_func("JSON_PARSE"), exp.ParseJSON: rename_func("JSON_PARSE"),
exp.SafeConcat: concat_to_dpipe_sql,
exp.Select: transforms.preprocess( exp.Select: transforms.preprocess(
[transforms.eliminate_distinct_on, transforms.eliminate_semi_and_anti_joins] [transforms.eliminate_distinct_on, transforms.eliminate_semi_and_anti_joins]
), ),
exp.SortKeyProperty: lambda self, e: f"{'COMPOUND ' if e.args['compound'] else ''}SORTKEY({self.format_args(*e.this)})", exp.SortKeyProperty: lambda self, e: f"{'COMPOUND ' if e.args['compound'] else ''}SORTKEY({self.format_args(*e.this)})",
exp.TsOrDsAdd: date_delta_sql("DATEADD"),
exp.TsOrDsDiff: date_delta_sql("DATEDIFF"),
exp.TsOrDsToDate: ts_or_ds_to_date_sql("redshift"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("redshift"),
} }

113
sqlglot/dialects/snowflake.py
View file

@ -3,9 +3,12 @@ from __future__ import annotations
import typing as t import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot._typing import E
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
binary_from_function, binary_from_function,
date_delta_sql,
date_trunc_to_time, date_trunc_to_time,
datestrtodate_sql, datestrtodate_sql,
format_time_lambda, format_time_lambda,
@ -21,7 +24,6 @@ from sqlglot.dialects.dialect import (
) )
from sqlglot.expressions import Literal from sqlglot.expressions import Literal
from sqlglot.helper import seq_get from sqlglot.helper import seq_get
from sqlglot.parser import binary_range_parser
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
@ -50,7 +52,7 @@ def _parse_to_timestamp(args: t.List) -> t.Union[exp.StrToTime, exp.UnixToTime,
elif second_arg.name == "3": elif second_arg.name == "3":
timescale = exp.UnixToTime.MILLIS timescale = exp.UnixToTime.MILLIS
elif second_arg.name == "9": elif second_arg.name == "9":
timescale = exp.UnixToTime.MICROS timescale = exp.UnixToTime.NANOS
return exp.UnixToTime(this=first_arg, scale=timescale) return exp.UnixToTime(this=first_arg, scale=timescale)
@ -95,14 +97,17 @@ def _parse_datediff(args: t.List) -> exp.DateDiff:
def _unix_to_time_sql(self: Snowflake.Generator, expression: exp.UnixToTime) -> str: def _unix_to_time_sql(self: Snowflake.Generator, expression: exp.UnixToTime) -> str:
scale = expression.args.get("scale") scale = expression.args.get("scale")
timestamp = self.sql(expression, "this") timestamp = self.sql(expression, "this")
if scale in [None, exp.UnixToTime.SECONDS]: if scale in (None, exp.UnixToTime.SECONDS):
return f"TO_TIMESTAMP({timestamp})" return f"TO_TIMESTAMP({timestamp})"
if scale == exp.UnixToTime.MILLIS: if scale == exp.UnixToTime.MILLIS:
return f"TO_TIMESTAMP({timestamp}, 3)" return f"TO_TIMESTAMP({timestamp}, 3)"
if scale == exp.UnixToTime.MICROS: if scale == exp.UnixToTime.MICROS:
return f"TO_TIMESTAMP({timestamp} / 1000, 3)"
if scale == exp.UnixToTime.NANOS:
return f"TO_TIMESTAMP({timestamp}, 9)" return f"TO_TIMESTAMP({timestamp}, 9)"
raise ValueError("Improper scale for timestamp") self.unsupported(f"Unsupported scale for timestamp: {scale}.")
return ""
# https://docs.snowflake.com/en/sql-reference/functions/date_part.html # https://docs.snowflake.com/en/sql-reference/functions/date_part.html
@ -201,7 +206,7 @@ def _show_parser(*args: t.Any, **kwargs: t.Any) -> t.Callable[[Snowflake.Parser]
class Snowflake(Dialect): class Snowflake(Dialect):
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax # https://docs.snowflake.com/en/sql-reference/identifiers-syntax
RESOLVES_IDENTIFIERS_AS_UPPERCASE = True NORMALIZATION_STRATEGY = NormalizationStrategy.UPPERCASE
NULL_ORDERING = "nulls_are_large" NULL_ORDERING = "nulls_are_large"
TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'" TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'"
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
@ -236,6 +241,18 @@ class Snowflake(Dialect):
"ff6": "%f", "ff6": "%f",
} }
def quote_identifier(self, expression: E, identify: bool = True) -> E:
# This disables quoting DUAL in SELECT ... FROM DUAL, because Snowflake treats an
# unquoted DUAL keyword in a special way and does not map it to a user-defined table
if (
isinstance(expression, exp.Identifier)
and isinstance(expression.parent, exp.Table)
and expression.name.lower() == "dual"
):
return t.cast(E, expression)
return super().quote_identifier(expression, identify=identify)
class Parser(parser.Parser): class Parser(parser.Parser):
IDENTIFY_PIVOT_STRINGS = True IDENTIFY_PIVOT_STRINGS = True
@ -245,6 +262,9 @@ class Snowflake(Dialect):
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,
"ARRAYAGG": exp.ArrayAgg.from_arg_list, "ARRAYAGG": exp.ArrayAgg.from_arg_list,
"ARRAY_CONSTRUCT": exp.Array.from_arg_list, "ARRAY_CONSTRUCT": exp.Array.from_arg_list,
"ARRAY_CONTAINS": lambda args: exp.ArrayContains(
this=seq_get(args, 1), expression=seq_get(args, 0)
),
"ARRAY_GENERATE_RANGE": lambda args: exp.GenerateSeries( "ARRAY_GENERATE_RANGE": lambda args: exp.GenerateSeries(
# ARRAY_GENERATE_RANGE has an exlusive end; we normalize it to be inclusive # ARRAY_GENERATE_RANGE has an exlusive end; we normalize it to be inclusive
start=seq_get(args, 0), start=seq_get(args, 0),
@ -296,8 +316,8 @@ class Snowflake(Dialect):
RANGE_PARSERS = { RANGE_PARSERS = {
**parser.Parser.RANGE_PARSERS, **parser.Parser.RANGE_PARSERS,
TokenType.LIKE_ANY: binary_range_parser(exp.LikeAny), TokenType.LIKE_ANY: parser.binary_range_parser(exp.LikeAny),
TokenType.ILIKE_ANY: binary_range_parser(exp.ILikeAny), TokenType.ILIKE_ANY: parser.binary_range_parser(exp.ILikeAny),
} }
ALTER_PARSERS = { ALTER_PARSERS = {
@ -317,6 +337,11 @@ class Snowflake(Dialect):
TokenType.SHOW: lambda self: self._parse_show(), TokenType.SHOW: lambda self: self._parse_show(),
} }
PROPERTY_PARSERS = {
**parser.Parser.PROPERTY_PARSERS,
"LOCATION": lambda self: self._parse_location(),
}
SHOW_PARSERS = { SHOW_PARSERS = {
"PRIMARY KEYS": _show_parser("PRIMARY KEYS"), "PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
"TERSE PRIMARY KEYS": _show_parser("PRIMARY KEYS"), "TERSE PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
@ -349,7 +374,7 @@ class Snowflake(Dialect):
table: t.Optional[exp.Expression] = None table: t.Optional[exp.Expression] = None
if self._match_text_seq("@"): if self._match_text_seq("@"):
table_name = "@" table_name = "@"
while True: while self._curr:
self._advance() self._advance()
table_name += self._prev.text table_name += self._prev.text
if not self._match_set(self.STAGED_FILE_SINGLE_TOKENS, advance=False): if not self._match_set(self.STAGED_FILE_SINGLE_TOKENS, advance=False):
@ -411,6 +436,20 @@ class Snowflake(Dialect):
self._match_text_seq("WITH") self._match_text_seq("WITH")
return self.expression(exp.SwapTable, this=self._parse_table(schema=True)) return self.expression(exp.SwapTable, this=self._parse_table(schema=True))
def _parse_location(self) -> exp.LocationProperty:
self._match(TokenType.EQ)
parts = [self._parse_var(any_token=True)]
while self._match(TokenType.SLASH):
if self._curr and self._prev.end + 1 == self._curr.start:
parts.append(self._parse_var(any_token=True))
else:
parts.append(exp.Var(this=""))
return self.expression(
exp.LocationProperty, this=exp.var("/".join(str(p) for p in parts))
)
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
STRING_ESCAPES = ["\\", "'"] STRING_ESCAPES = ["\\", "'"]
HEX_STRINGS = [("x'", "'"), ("X'", "'")] HEX_STRINGS = [("x'", "'"), ("X'", "'")]
@ -457,6 +496,7 @@ class Snowflake(Dialect):
AGGREGATE_FILTER_SUPPORTED = False AGGREGATE_FILTER_SUPPORTED = False
SUPPORTS_TABLE_COPY = False SUPPORTS_TABLE_COPY = False
COLLATE_IS_FUNC = True COLLATE_IS_FUNC = True
LIMIT_ONLY_LITERALS = True
TRANSFORMS = { TRANSFORMS = {
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,
@ -464,15 +504,14 @@ class Snowflake(Dialect):
exp.ArgMin: rename_func("MIN_BY"), exp.ArgMin: rename_func("MIN_BY"),
exp.Array: inline_array_sql, exp.Array: inline_array_sql,
exp.ArrayConcat: rename_func("ARRAY_CAT"), exp.ArrayConcat: rename_func("ARRAY_CAT"),
exp.ArrayContains: lambda self, e: self.func("ARRAY_CONTAINS", e.expression, e.this),
exp.ArrayJoin: rename_func("ARRAY_TO_STRING"), exp.ArrayJoin: rename_func("ARRAY_TO_STRING"),
exp.AtTimeZone: lambda self, e: self.func( exp.AtTimeZone: lambda self, e: self.func(
"CONVERT_TIMEZONE", e.args.get("zone"), e.this "CONVERT_TIMEZONE", e.args.get("zone"), e.this
), ),
exp.BitwiseXor: rename_func("BITXOR"), exp.BitwiseXor: rename_func("BITXOR"),
exp.DateAdd: lambda self, e: self.func("DATEADD", e.text("unit"), e.expression, e.this), exp.DateAdd: date_delta_sql("DATEADD"),
exp.DateDiff: lambda self, e: self.func( exp.DateDiff: date_delta_sql("DATEDIFF"),
"DATEDIFF", e.text("unit"), e.expression, e.this
),
exp.DateStrToDate: datestrtodate_sql, exp.DateStrToDate: datestrtodate_sql,
exp.DataType: _datatype_sql, exp.DataType: _datatype_sql,
exp.DayOfMonth: rename_func("DAYOFMONTH"), exp.DayOfMonth: rename_func("DAYOFMONTH"),
@ -501,10 +540,11 @@ class Snowflake(Dialect):
exp.Select: transforms.preprocess( exp.Select: transforms.preprocess(
[ [
transforms.eliminate_distinct_on, transforms.eliminate_distinct_on,
transforms.explode_to_unnest(0), transforms.explode_to_unnest(),
transforms.eliminate_semi_and_anti_joins, transforms.eliminate_semi_and_anti_joins,
] ]
), ),
exp.SHA: rename_func("SHA1"),
exp.StarMap: rename_func("OBJECT_CONSTRUCT"), exp.StarMap: rename_func("OBJECT_CONSTRUCT"),
exp.StartsWith: rename_func("STARTSWITH"), exp.StartsWith: rename_func("STARTSWITH"),
exp.StrPosition: lambda self, e: self.func( exp.StrPosition: lambda self, e: self.func(
@ -524,6 +564,8 @@ class Snowflake(Dialect):
exp.TimeToUnix: lambda self, e: f"EXTRACT(epoch_second FROM {self.sql(e, 'this')})", exp.TimeToUnix: lambda self, e: f"EXTRACT(epoch_second FROM {self.sql(e, 'this')})",
exp.ToChar: lambda self, e: self.function_fallback_sql(e), exp.ToChar: lambda self, e: self.function_fallback_sql(e),
exp.Trim: lambda self, e: self.func("TRIM", e.this, e.expression), exp.Trim: lambda self, e: self.func("TRIM", e.this, e.expression),
exp.TsOrDsAdd: date_delta_sql("DATEADD", cast=True),
exp.TsOrDsDiff: date_delta_sql("DATEDIFF"),
exp.TsOrDsToDate: ts_or_ds_to_date_sql("snowflake"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("snowflake"),
exp.UnixToTime: _unix_to_time_sql, exp.UnixToTime: _unix_to_time_sql,
exp.VarMap: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"), exp.VarMap: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"),
@ -547,6 +589,20 @@ class Snowflake(Dialect):
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED, exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
} }
def trycast_sql(self, expression: exp.TryCast) -> str:
value = expression.this
if value.type is None:
from sqlglot.optimizer.annotate_types import annotate_types
value = annotate_types(value)
if value.is_type(*exp.DataType.TEXT_TYPES, exp.DataType.Type.UNKNOWN):
return super().trycast_sql(expression)
# TRY_CAST only works for string values in Snowflake
return self.cast_sql(expression)
def log_sql(self, expression: exp.Log) -> str: def log_sql(self, expression: exp.Log) -> str:
if not expression.expression: if not expression.expression:
return self.func("LN", expression.this) return self.func("LN", expression.this)
@ -554,24 +610,28 @@ class Snowflake(Dialect):
return super().log_sql(expression) return super().log_sql(expression)
def unnest_sql(self, expression: exp.Unnest) -> str: def unnest_sql(self, expression: exp.Unnest) -> str:
selects = ["value"]
unnest_alias = expression.args.get("alias") unnest_alias = expression.args.get("alias")
offset = expression.args.get("offset") offset = expression.args.get("offset")
if offset:
columns = [
exp.to_identifier("seq"),
exp.to_identifier("key"),
exp.to_identifier("path"),
offset.pop() if isinstance(offset, exp.Expression) else exp.to_identifier("index"),
seq_get(unnest_alias.columns if unnest_alias else [], 0)
or exp.to_identifier("value"),
exp.to_identifier("this"),
]
if unnest_alias: if unnest_alias:
unnest_alias.append("columns", offset.pop()) unnest_alias.set("columns", columns)
else:
unnest_alias = exp.TableAlias(this="_u", columns=columns)
selects.append("index") explode = f"TABLE(FLATTEN(INPUT => {self.sql(expression.expressions[0])}))"
subquery = exp.Subquery(
this=exp.select(*selects).from_(
f"TABLE(FLATTEN(INPUT => {self.sql(expression.expressions[0])}))"
),
)
alias = self.sql(unnest_alias) alias = self.sql(unnest_alias)
alias = f" AS {alias}" if alias else "" alias = f" AS {alias}" if alias else ""
return f"{self.sql(subquery)}{alias}" return f"{explode}{alias}"
def show_sql(self, expression: exp.Show) -> str: def show_sql(self, expression: exp.Show) -> str:
scope = self.sql(expression, "scope") scope = self.sql(expression, "scope")
@ -632,3 +692,6 @@ class Snowflake(Dialect):
def swaptable_sql(self, expression: exp.SwapTable) -> str: def swaptable_sql(self, expression: exp.SwapTable) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
return f"SWAP WITH {this}" return f"SWAP WITH {this}"
def with_properties(self, properties: exp.Properties) -> str:
return self.properties(properties, wrapped=False, prefix=self.seg(""), sep=" ")

8
sqlglot/dialects/spark.py
View file

@ -56,15 +56,17 @@ class Spark(Spark2):
def _parse_generated_as_identity( def _parse_generated_as_identity(
self, self,
) -> exp.GeneratedAsIdentityColumnConstraint | exp.ComputedColumnConstraint: ) -> (
exp.GeneratedAsIdentityColumnConstraint
| exp.ComputedColumnConstraint
| exp.GeneratedAsRowColumnConstraint
):
this = super()._parse_generated_as_identity() this = super()._parse_generated_as_identity()
if this.expression: if this.expression:
return self.expression(exp.ComputedColumnConstraint, this=this.expression) return self.expression(exp.ComputedColumnConstraint, this=this.expression)
return this return this
class Generator(Spark2.Generator): class Generator(Spark2.Generator):
SUPPORTS_NESTED_CTES = True
TYPE_MAPPING = { TYPE_MAPPING = {
**Spark2.Generator.TYPE_MAPPING, **Spark2.Generator.TYPE_MAPPING,
exp.DataType.Type.MONEY: "DECIMAL(15, 4)", exp.DataType.Type.MONEY: "DECIMAL(15, 4)",

24
sqlglot/dialects/spark2.py
View file

@ -48,8 +48,11 @@ def _unix_to_time_sql(self: Spark2.Generator, expression: exp.UnixToTime) -> str
return f"TIMESTAMP_MILLIS({timestamp})" return f"TIMESTAMP_MILLIS({timestamp})"
if scale == exp.UnixToTime.MICROS: if scale == exp.UnixToTime.MICROS:
return f"TIMESTAMP_MICROS({timestamp})" return f"TIMESTAMP_MICROS({timestamp})"
if scale == exp.UnixToTime.NANOS:
return f"TIMESTAMP_SECONDS({timestamp} / 1000000000)"
raise ValueError("Improper scale for timestamp") self.unsupported(f"Unsupported scale for timestamp: {scale}.")
return ""
def _unalias_pivot(expression: exp.Expression) -> exp.Expression: def _unalias_pivot(expression: exp.Expression) -> exp.Expression:
@ -119,7 +122,11 @@ class Spark2(Hive):
"DOUBLE": _parse_as_cast("double"), "DOUBLE": _parse_as_cast("double"),
"FLOAT": _parse_as_cast("float"), "FLOAT": _parse_as_cast("float"),
"FROM_UTC_TIMESTAMP": lambda args: exp.AtTimeZone( "FROM_UTC_TIMESTAMP": lambda args: exp.AtTimeZone(
this=exp.Cast(this=seq_get(args, 0), to=exp.DataType.build("timestamp")), this=exp.cast_unless(
seq_get(args, 0) or exp.Var(this=""),
exp.DataType.build("timestamp"),
exp.DataType.build("timestamp"),
),
zone=seq_get(args, 1), zone=seq_get(args, 1),
), ),
"IIF": exp.If.from_arg_list, "IIF": exp.If.from_arg_list,
@ -224,6 +231,19 @@ class Spark2(Hive):
WRAP_DERIVED_VALUES = False WRAP_DERIVED_VALUES = False
CREATE_FUNCTION_RETURN_AS = False CREATE_FUNCTION_RETURN_AS = False
def struct_sql(self, expression: exp.Struct) -> str:
args = []
for arg in expression.expressions:
if isinstance(arg, self.KEY_VALUE_DEFINITONS):
if isinstance(arg, exp.Bracket):
args.append(exp.alias_(arg.this, arg.expressions[0].name))
else:
args.append(exp.alias_(arg.expression, arg.this.name))
else:
args.append(arg)
return self.func("STRUCT", *args)
def temporary_storage_provider(self, expression: exp.Create) -> exp.Create: def temporary_storage_provider(self, expression: exp.Create) -> exp.Create:
# spark2, spark, Databricks require a storage provider for temporary tables # spark2, spark, Databricks require a storage provider for temporary tables
provider = exp.FileFormatProperty(this=exp.Literal.string("parquet")) provider = exp.FileFormatProperty(this=exp.Literal.string("parquet"))

6
sqlglot/dialects/sqlite.py
View file

@ -5,6 +5,7 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
any_value_to_max_sql, any_value_to_max_sql,
arrow_json_extract_scalar_sql, arrow_json_extract_scalar_sql,
arrow_json_extract_sql, arrow_json_extract_sql,
@ -63,8 +64,10 @@ def _transform_create(expression: exp.Expression) -> exp.Expression:
class SQLite(Dialect): class SQLite(Dialect):
# https://sqlite.org/forum/forumpost/5e575586ac5c711b?raw # https://sqlite.org/forum/forumpost/5e575586ac5c711b?raw
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
TYPED_DIVISION = True
SAFE_DIVISION = True
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
IDENTIFIERS = ['"', ("[", "]"), "`"] IDENTIFIERS = ['"', ("[", "]"), "`"]
@ -124,7 +127,6 @@ class SQLite(Dialect):
exp.LogicalOr: rename_func("MAX"), exp.LogicalOr: rename_func("MAX"),
exp.LogicalAnd: rename_func("MIN"), exp.LogicalAnd: rename_func("MIN"),
exp.Pivot: no_pivot_sql, exp.Pivot: no_pivot_sql,
exp.SafeConcat: concat_to_dpipe_sql,
exp.Select: transforms.preprocess( exp.Select: transforms.preprocess(
[ [
transforms.eliminate_distinct_on, transforms.eliminate_distinct_on,

13
sqlglot/dialects/teradata.py
View file

@ -9,6 +9,7 @@ from sqlglot.tokens import TokenType
class Teradata(Dialect): class Teradata(Dialect):
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
TYPED_DIVISION = True
TIME_MAPPING = { TIME_MAPPING = {
"Y": "%Y", "Y": "%Y",
@ -33,8 +34,10 @@ class Teradata(Dialect):
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
# https://docs.teradata.com/r/Teradata-Database-SQL-Functions-Operators-Expressions-and-Predicates/March-2017/Comparison-Operators-and-Functions/Comparison-Operators/ANSI-Compliance # https://docs.teradata.com/r/Teradata-Database-SQL-Functions-Operators-Expressions-and-Predicates/March-2017/Comparison-Operators-and-Functions/Comparison-Operators/ANSI-Compliance
# https://docs.teradata.com/r/SQL-Functions-Operators-Expressions-and-Predicates/June-2017/Arithmetic-Trigonometric-Hyperbolic-Operators/Functions
KEYWORDS = { KEYWORDS = {
**tokens.Tokenizer.KEYWORDS, **tokens.Tokenizer.KEYWORDS,
"**": TokenType.DSTAR,
"^=": TokenType.NEQ, "^=": TokenType.NEQ,
"BYTEINT": TokenType.SMALLINT, "BYTEINT": TokenType.SMALLINT,
"COLLECT": TokenType.COMMAND, "COLLECT": TokenType.COMMAND,
@ -112,10 +115,16 @@ class Teradata(Dialect):
FUNCTION_PARSERS = { FUNCTION_PARSERS = {
**parser.Parser.FUNCTION_PARSERS, **parser.Parser.FUNCTION_PARSERS,
# https://docs.teradata.com/r/SQL-Functions-Operators-Expressions-and-Predicates/June-2017/Data-Type-Conversions/TRYCAST
"TRYCAST": parser.Parser.FUNCTION_PARSERS["TRY_CAST"],
"RANGE_N": lambda self: self._parse_rangen(), "RANGE_N": lambda self: self._parse_rangen(),
"TRANSLATE": lambda self: self._parse_translate(self.STRICT_CAST), "TRANSLATE": lambda self: self._parse_translate(self.STRICT_CAST),
} }
EXPONENT = {
TokenType.DSTAR: exp.Pow,
}
def _parse_translate(self, strict: bool) -> exp.Expression: def _parse_translate(self, strict: bool) -> exp.Expression:
this = self._parse_conjunction() this = self._parse_conjunction()
@ -177,6 +186,7 @@ class Teradata(Dialect):
exp.ArgMin: rename_func("MIN_BY"), exp.ArgMin: rename_func("MIN_BY"),
exp.Max: max_or_greatest, exp.Max: max_or_greatest,
exp.Min: min_or_least, exp.Min: min_or_least,
exp.Pow: lambda self, e: self.binary(e, "**"),
exp.Select: transforms.preprocess( exp.Select: transforms.preprocess(
[transforms.eliminate_distinct_on, transforms.eliminate_semi_and_anti_joins] [transforms.eliminate_distinct_on, transforms.eliminate_semi_and_anti_joins]
), ),
@ -192,6 +202,9 @@ class Teradata(Dialect):
return super().cast_sql(expression, safe_prefix=safe_prefix) return super().cast_sql(expression, safe_prefix=safe_prefix)
def trycast_sql(self, expression: exp.TryCast) -> str:
return self.cast_sql(expression, safe_prefix="TRY")
def tablesample_sql( def tablesample_sql(
self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS " self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS "
) -> str: ) -> str:

118
sqlglot/dialects/tsql.py
View file

@ -7,7 +7,9 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
Dialect, Dialect,
NormalizationStrategy,
any_value_to_max_sql, any_value_to_max_sql,
date_delta_sql,
generatedasidentitycolumnconstraint_sql, generatedasidentitycolumnconstraint_sql,
max_or_greatest, max_or_greatest,
min_or_least, min_or_least,
@ -135,11 +137,7 @@ def _parse_hashbytes(args: t.List) -> exp.Expression:
return exp.func("HASHBYTES", *args) return exp.func("HASHBYTES", *args)
def generate_date_delta_with_unit_sql( DATEPART_ONLY_FORMATS = {"dw", "hour", "quarter"}
self: TSQL.Generator, expression: exp.DateAdd | exp.DateDiff
) -> str:
func = "DATEADD" if isinstance(expression, exp.DateAdd) else "DATEDIFF"
return self.func(func, expression.text("unit"), expression.expression, expression.this)
def _format_sql(self: TSQL.Generator, expression: exp.NumberToStr | exp.TimeToStr) -> str: def _format_sql(self: TSQL.Generator, expression: exp.NumberToStr | exp.TimeToStr) -> str:
@ -153,6 +151,11 @@ def _format_sql(self: TSQL.Generator, expression: exp.NumberToStr | exp.TimeToSt
) )
) )
) )
# There is no format for "quarter"
if fmt.name.lower() in DATEPART_ONLY_FORMATS:
return self.func("DATEPART", fmt.name, expression.this)
return self.func("FORMAT", expression.this, fmt, expression.args.get("culture")) return self.func("FORMAT", expression.this, fmt, expression.args.get("culture"))
@ -202,18 +205,50 @@ def _parse_date_delta(
return inner_func return inner_func
def qualify_derived_table_outputs(expression: exp.Expression) -> exp.Expression:
"""Ensures all (unnamed) output columns are aliased for CTEs and Subqueries."""
alias = expression.args.get("alias")
if (
isinstance(expression, (exp.CTE, exp.Subquery))
and isinstance(alias, exp.TableAlias)
and not alias.columns
):
from sqlglot.optimizer.qualify_columns import qualify_outputs
# We keep track of the unaliased column projection indexes instead of the expressions
# themselves, because the latter are going to be replaced by new nodes when the aliases
# are added and hence we won't be able to reach these newly added Alias parents
subqueryable = expression.this
unaliased_column_indexes = (
i
for i, c in enumerate(subqueryable.selects)
if isinstance(c, exp.Column) and not c.alias
)
qualify_outputs(subqueryable)
# Preserve the quoting information of columns for newly added Alias nodes
subqueryable_selects = subqueryable.selects
for select_index in unaliased_column_indexes:
alias = subqueryable_selects[select_index]
column = alias.this
if isinstance(column.this, exp.Identifier):
alias.args["alias"].set("quoted", column.this.quoted)
return expression
class TSQL(Dialect): class TSQL(Dialect):
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
NULL_ORDERING = "nulls_are_small"
TIME_FORMAT = "'yyyy-mm-dd hh:mm:ss'" TIME_FORMAT = "'yyyy-mm-dd hh:mm:ss'"
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
LOG_BASE_FIRST = False LOG_BASE_FIRST = False
TYPED_DIVISION = True
CONCAT_COALESCE = True
TIME_MAPPING = { TIME_MAPPING = {
"year": "%Y", "year": "%Y",
"qq": "%q",
"q": "%q",
"quarter": "%q",
"dayofyear": "%j", "dayofyear": "%j",
"day": "%d", "day": "%d",
"dy": "%d", "dy": "%d",
@ -320,6 +355,7 @@ class TSQL(Dialect):
IDENTIFIERS = ['"', ("[", "]")] IDENTIFIERS = ['"', ("[", "]")]
QUOTES = ["'", '"'] QUOTES = ["'", '"']
HEX_STRINGS = [("0x", ""), ("0X", "")] HEX_STRINGS = [("0x", ""), ("0X", "")]
VAR_SINGLE_TOKENS = {"@", "$", "#"}
KEYWORDS = { KEYWORDS = {
**tokens.Tokenizer.KEYWORDS, **tokens.Tokenizer.KEYWORDS,
@ -403,9 +439,7 @@ class TSQL(Dialect):
LOG_DEFAULTS_TO_LN = True LOG_DEFAULTS_TO_LN = True
CONCAT_NULL_OUTPUTS_STRING = True ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = False
ALTER_TABLE_ADD_COLUMN_KEYWORD = False
def _parse_projections(self) -> t.List[exp.Expression]: def _parse_projections(self) -> t.List[exp.Expression]:
""" """
@ -433,7 +467,7 @@ class TSQL(Dialect):
""" """
rollback = self._prev.token_type == TokenType.ROLLBACK rollback = self._prev.token_type == TokenType.ROLLBACK
self._match_texts({"TRAN", "TRANSACTION"}) self._match_texts(("TRAN", "TRANSACTION"))
this = self._parse_id_var() this = self._parse_id_var()
if rollback: if rollback:
@ -579,23 +613,35 @@ class TSQL(Dialect):
return super()._parse_if() return super()._parse_if()
def _parse_unique(self) -> exp.UniqueColumnConstraint: def _parse_unique(self) -> exp.UniqueColumnConstraint:
return self.expression( if self._match_texts(("CLUSTERED", "NONCLUSTERED")):
exp.UniqueColumnConstraint, this = self.CONSTRAINT_PARSERS[self._prev.text.upper()](self)
this=None else:
if self._curr and self._curr.text.upper() in {"CLUSTERED", "NONCLUSTERED"} this = self._parse_schema(self._parse_id_var(any_token=False))
else self._parse_schema(self._parse_id_var(any_token=False)),
) return self.expression(exp.UniqueColumnConstraint, this=this)
class Generator(generator.Generator): class Generator(generator.Generator):
LIMIT_IS_TOP = True LIMIT_IS_TOP = True
QUERY_HINTS = False QUERY_HINTS = False
RETURNING_END = False RETURNING_END = False
NVL2_SUPPORTED = False NVL2_SUPPORTED = False
ALTER_TABLE_ADD_COLUMN_KEYWORD = False ALTER_TABLE_INCLUDE_COLUMN_KEYWORD = False
LIMIT_FETCH = "FETCH" LIMIT_FETCH = "FETCH"
COMPUTED_COLUMN_WITH_TYPE = False COMPUTED_COLUMN_WITH_TYPE = False
SUPPORTS_NESTED_CTES = False
CTE_RECURSIVE_KEYWORD_REQUIRED = False CTE_RECURSIVE_KEYWORD_REQUIRED = False
ENSURE_BOOLS = True
NULL_ORDERING_SUPPORTED = False
SUPPORTS_SINGLE_ARG_CONCAT = False
EXPRESSIONS_WITHOUT_NESTED_CTES = {
exp.Delete,
exp.Insert,
exp.Merge,
exp.Select,
exp.Subquery,
exp.Union,
exp.Update,
}
TYPE_MAPPING = { TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING, **generator.Generator.TYPE_MAPPING,
@ -614,14 +660,16 @@ class TSQL(Dialect):
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,
exp.AnyValue: any_value_to_max_sql, exp.AnyValue: any_value_to_max_sql,
exp.AutoIncrementColumnConstraint: lambda *_: "IDENTITY", exp.AutoIncrementColumnConstraint: lambda *_: "IDENTITY",
exp.DateAdd: generate_date_delta_with_unit_sql, exp.DateAdd: date_delta_sql("DATEADD"),
exp.DateDiff: generate_date_delta_with_unit_sql, exp.DateDiff: date_delta_sql("DATEDIFF"),
exp.CTE: transforms.preprocess([qualify_derived_table_outputs]),
exp.CurrentDate: rename_func("GETDATE"), exp.CurrentDate: rename_func("GETDATE"),
exp.CurrentTimestamp: rename_func("GETDATE"), exp.CurrentTimestamp: rename_func("GETDATE"),
exp.Extract: rename_func("DATEPART"), exp.Extract: rename_func("DATEPART"),
exp.GeneratedAsIdentityColumnConstraint: generatedasidentitycolumnconstraint_sql, exp.GeneratedAsIdentityColumnConstraint: generatedasidentitycolumnconstraint_sql,
exp.GroupConcat: _string_agg_sql, exp.GroupConcat: _string_agg_sql,
exp.If: rename_func("IIF"), exp.If: rename_func("IIF"),
exp.Length: rename_func("LEN"),
exp.Max: max_or_greatest, exp.Max: max_or_greatest,
exp.MD5: lambda self, e: self.func("HASHBYTES", exp.Literal.string("MD5"), e.this), exp.MD5: lambda self, e: self.func("HASHBYTES", exp.Literal.string("MD5"), e.this),
exp.Min: min_or_least, exp.Min: min_or_least,
@ -633,15 +681,16 @@ class TSQL(Dialect):
transforms.eliminate_qualify, transforms.eliminate_qualify,
] ]
), ),
exp.Subquery: transforms.preprocess([qualify_derived_table_outputs]),
exp.SHA: lambda self, e: self.func("HASHBYTES", exp.Literal.string("SHA1"), e.this), exp.SHA: lambda self, e: self.func("HASHBYTES", exp.Literal.string("SHA1"), e.this),
exp.SHA2: lambda self, e: self.func( exp.SHA2: lambda self, e: self.func(
"HASHBYTES", "HASHBYTES", exp.Literal.string(f"SHA2_{e.args.get('length', 256)}"), e.this
exp.Literal.string(f"SHA2_{e.args.get('length', 256)}"),
e.this,
), ),
exp.TemporaryProperty: lambda self, e: "", exp.TemporaryProperty: lambda self, e: "",
exp.TimeStrToTime: timestrtotime_sql, exp.TimeStrToTime: timestrtotime_sql,
exp.TimeToStr: _format_sql, exp.TimeToStr: _format_sql,
exp.TsOrDsAdd: date_delta_sql("DATEADD", cast=True),
exp.TsOrDsDiff: date_delta_sql("DATEDIFF"),
exp.TsOrDsToDate: ts_or_ds_to_date_sql("tsql"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("tsql"),
} }
@ -690,8 +739,21 @@ class TSQL(Dialect):
table = expression.find(exp.Table) table = expression.find(exp.Table)
# Convert CTAS statement to SELECT .. INTO ..
if kind == "TABLE" and expression.expression: if kind == "TABLE" and expression.expression:
sql = f"SELECT * INTO {self.sql(table)} FROM ({self.sql(expression.expression)}) AS temp" ctas_with = expression.expression.args.get("with")
if ctas_with:
ctas_with = ctas_with.pop()
subquery = expression.expression
if isinstance(subquery, exp.Subqueryable):
subquery = subquery.subquery()
select_into = exp.select("*").from_(exp.alias_(subquery, "temp", table=True))
select_into.set("into", exp.Into(this=table))
select_into.set("with", ctas_with)
sql = self.sql(select_into)
if exists: if exists:
identifier = self.sql(exp.Literal.string(exp.table_name(table) if table else "")) identifier = self.sql(exp.Literal.string(exp.table_name(table) if table else ""))

12
sqlglot/executor/env.py
View file

@ -139,10 +139,16 @@ def interval(this, unit):
return datetime.timedelta(**{unit: float(this)}) return datetime.timedelta(**{unit: float(this)})
@null_if_any("this", "expression")
def arrayjoin(this, expression, null=None):
return expression.join(x for x in (x if x is not None else null for x in this) if x is not None)
ENV = { ENV = {
"exp": exp, "exp": exp,
# aggs # aggs
"ARRAYAGG": list, "ARRAYAGG": list,
"ARRAYUNIQUEAGG": filter_nulls(lambda acc: list(set(acc))),
"AVG": filter_nulls(statistics.fmean if PYTHON_VERSION >= (3, 8) else statistics.mean), # type: ignore "AVG": filter_nulls(statistics.fmean if PYTHON_VERSION >= (3, 8) else statistics.mean), # type: ignore
"COUNT": filter_nulls(lambda acc: sum(1 for _ in acc), False), "COUNT": filter_nulls(lambda acc: sum(1 for _ in acc), False),
"MAX": filter_nulls(max), "MAX": filter_nulls(max),
@ -152,6 +158,7 @@ ENV = {
"ABS": null_if_any(lambda this: abs(this)), "ABS": null_if_any(lambda this: abs(this)),
"ADD": null_if_any(lambda e, this: e + this), "ADD": null_if_any(lambda e, this: e + this),
"ARRAYANY": null_if_any(lambda arr, func: any(func(e) for e in arr)), "ARRAYANY": null_if_any(lambda arr, func: any(func(e) for e in arr)),
"ARRAYJOIN": arrayjoin,
"BETWEEN": null_if_any(lambda this, low, high: low <= this and this <= high), "BETWEEN": null_if_any(lambda this, low, high: low <= this and this <= high),
"BITWISEAND": null_if_any(lambda this, e: this & e), "BITWISEAND": null_if_any(lambda this, e: this & e),
"BITWISELEFTSHIFT": null_if_any(lambda this, e: this << e), "BITWISELEFTSHIFT": null_if_any(lambda this, e: this << e),
@ -203,4 +210,9 @@ ENV = {
"CURRENTDATE": datetime.date.today, "CURRENTDATE": datetime.date.today,
"STRFTIME": null_if_any(lambda fmt, arg: datetime.datetime.fromisoformat(arg).strftime(fmt)), "STRFTIME": null_if_any(lambda fmt, arg: datetime.datetime.fromisoformat(arg).strftime(fmt)),
"TRIM": null_if_any(lambda this, e=None: this.strip(e)), "TRIM": null_if_any(lambda this, e=None: this.strip(e)),
"STRUCT": lambda *args: {
args[x]: args[x + 1]
for x in range(0, len(args), 2)
if (args[x + 1] is not None and args[x] is not None)
},
} }

18
sqlglot/executor/python.py
View file

@ -397,6 +397,20 @@ def _lambda_sql(self, e: exp.Lambda) -> str:
return f"lambda {self.expressions(e, flat=True)}: {self.sql(e, 'this')}" return f"lambda {self.expressions(e, flat=True)}: {self.sql(e, 'this')}"
def _div_sql(self: generator.Generator, e: exp.Div) -> str:
denominator = self.sql(e, "expression")
if e.args.get("safe"):
denominator += " or None"
sql = f"DIV({self.sql(e, 'this')}, {denominator})"
if e.args.get("typed"):
sql = f"int({sql})"
return sql
class Python(Dialect): class Python(Dialect):
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
STRING_ESCAPES = ["\\"] STRING_ESCAPES = ["\\"]
@ -413,7 +427,11 @@ class Python(Dialect):
exp.Boolean: lambda self, e: "True" if e.this else "False", exp.Boolean: lambda self, e: "True" if e.this else "False",
exp.Cast: lambda self, e: f"CAST({self.sql(e.this)}, exp.DataType.Type.{e.args['to']})", exp.Cast: lambda self, e: f"CAST({self.sql(e.this)}, exp.DataType.Type.{e.args['to']})",
exp.Column: lambda self, e: f"scope[{self.sql(e, 'table') or None}][{self.sql(e.this)}]", exp.Column: lambda self, e: f"scope[{self.sql(e, 'table') or None}][{self.sql(e.this)}]",
exp.Concat: lambda self, e: self.func(
"SAFECONCAT" if e.args.get("safe") else "CONCAT", *e.expressions
),
exp.Distinct: lambda self, e: f"set({self.sql(e, 'this')})", exp.Distinct: lambda self, e: f"set({self.sql(e, 'this')})",
exp.Div: _div_sql,
exp.Extract: lambda self, e: f"EXTRACT('{e.name.lower()}', {self.sql(e, 'expression')})", exp.Extract: lambda self, e: f"EXTRACT('{e.name.lower()}', {self.sql(e, 'expression')})",
exp.In: lambda self, e: f"{self.sql(e, 'this')} in {{{self.expressions(e, flat=True)}}}", exp.In: lambda self, e: f"{self.sql(e, 'this')} in {{{self.expressions(e, flat=True)}}}",
exp.Interval: lambda self, e: f"INTERVAL({self.sql(e.this)}, '{self.sql(e.unit)}')", exp.Interval: lambda self, e: f"INTERVAL({self.sql(e.this)}, '{self.sql(e.unit)}')",

8
sqlglot/executor/table.py
View file

@ -120,20 +120,22 @@ def _ensure_tables(d: t.Optional[t.Dict], dialect: DialectType = None) -> t.Dict
depth = dict_depth(d) depth = dict_depth(d)
if depth > 1: if depth > 1:
return { return {
normalize_name(k, dialect=dialect, is_table=True): _ensure_tables(v, dialect=dialect) normalize_name(k, dialect=dialect, is_table=True).name: _ensure_tables(
v, dialect=dialect
)
for k, v in d.items() for k, v in d.items()
} }
result = {} result = {}
for table_name, table in d.items(): for table_name, table in d.items():
table_name = normalize_name(table_name, dialect=dialect) table_name = normalize_name(table_name, dialect=dialect).name
if isinstance(table, Table): if isinstance(table, Table):
result[table_name] = table result[table_name] = table
else: else:
table = [ table = [
{ {
normalize_name(column_name, dialect=dialect): value normalize_name(column_name, dialect=dialect).name: value
for column_name, value in row.items() for column_name, value in row.items()
} }
for row in table for row in table

362
sqlglot/expressions.py
View file

@ -53,6 +53,7 @@ class _Expression(type):
SQLGLOT_META = "sqlglot.meta" SQLGLOT_META = "sqlglot.meta"
TABLE_PARTS = ("this", "db", "catalog")
class Expression(metaclass=_Expression): class Expression(metaclass=_Expression):
@ -134,7 +135,7 @@ class Expression(metaclass=_Expression):
return self.args.get("expression") return self.args.get("expression")
@property @property
def expressions(self): def expressions(self) -> t.List[t.Any]:
""" """
Retrieves the argument with key "expressions". Retrieves the argument with key "expressions".
""" """
@ -238,6 +239,9 @@ class Expression(metaclass=_Expression):
dtype = DataType.build(dtype) dtype = DataType.build(dtype)
self._type = dtype # type: ignore self._type = dtype # type: ignore
def is_type(self, *dtypes) -> bool:
return self.type is not None and self.type.is_type(*dtypes)
@property @property
def meta(self) -> t.Dict[str, t.Any]: def meta(self) -> t.Dict[str, t.Any]:
if self._meta is None: if self._meta is None:
@ -481,7 +485,7 @@ class Expression(metaclass=_Expression):
def flatten(self, unnest=True): def flatten(self, unnest=True):
""" """
Returns a generator which yields child nodes who's parents are the same class. Returns a generator which yields child nodes whose parents are the same class.
A AND B AND C -> [A, B, C] A AND B AND C -> [A, B, C]
""" """
@ -508,7 +512,7 @@ class Expression(metaclass=_Expression):
""" """
from sqlglot.dialects import Dialect from sqlglot.dialects import Dialect
return Dialect.get_or_raise(dialect)().generate(self, **opts) return Dialect.get_or_raise(dialect).generate(self, **opts)
def _to_s(self, hide_missing: bool = True, level: int = 0) -> str: def _to_s(self, hide_missing: bool = True, level: int = 0) -> str:
indent = "" if not level else "\n" indent = "" if not level else "\n"
@ -821,6 +825,12 @@ class Expression(metaclass=_Expression):
def rlike(self, other: ExpOrStr) -> RegexpLike: def rlike(self, other: ExpOrStr) -> RegexpLike:
return self._binop(RegexpLike, other) return self._binop(RegexpLike, other)
def div(self, other: ExpOrStr, typed: bool = False, safe: bool = False) -> Div:
div = self._binop(Div, other)
div.args["typed"] = typed
div.args["safe"] = safe
return div
def __lt__(self, other: t.Any) -> LT: def __lt__(self, other: t.Any) -> LT:
return self._binop(LT, other) return self._binop(LT, other)
@ -1000,7 +1010,6 @@ class UDTF(DerivedTable, Unionable):
class Cache(Expression): class Cache(Expression):
arg_types = { arg_types = {
"with": False,
"this": True, "this": True,
"lazy": False, "lazy": False,
"options": False, "options": False,
@ -1012,6 +1021,10 @@ class Uncache(Expression):
arg_types = {"this": True, "exists": False} arg_types = {"this": True, "exists": False}
class Refresh(Expression):
pass
class DDL(Expression): class DDL(Expression):
@property @property
def ctes(self): def ctes(self):
@ -1033,6 +1046,43 @@ class DDL(Expression):
return [] return []
class DML(Expression):
def returning(
self,
expression: ExpOrStr,
dialect: DialectType = None,
copy: bool = True,
**opts,
) -> DML:
"""
Set the RETURNING expression. Not supported by all dialects.
Example:
>>> delete("tbl").returning("*", dialect="postgres").sql()
'DELETE FROM tbl RETURNING *'
Args:
expression: the SQL code strings to parse.
If an `Expression` instance is passed, it will be used as-is.
dialect: the dialect used to parse the input expressions.
copy: if `False`, modify this expression instance in-place.
opts: other options to use to parse the input expressions.
Returns:
Delete: the modified expression.
"""
return _apply_builder(
expression=expression,
instance=self,
arg="returning",
prefix="RETURNING",
dialect=dialect,
copy=copy,
into=Returning,
**opts,
)
class Create(DDL): class Create(DDL):
arg_types = { arg_types = {
"with": False, "with": False,
@ -1133,8 +1183,10 @@ class WithinGroup(Expression):
arg_types = {"this": True, "expression": False} arg_types = {"this": True, "expression": False}
# clickhouse supports scalar ctes
# https://clickhouse.com/docs/en/sql-reference/statements/select/with
class CTE(DerivedTable): class CTE(DerivedTable):
arg_types = {"this": True, "alias": True} arg_types = {"this": True, "alias": True, "scalar": False}
class TableAlias(Expression): class TableAlias(Expression):
@ -1297,6 +1349,10 @@ class AutoIncrementColumnConstraint(ColumnConstraintKind):
pass pass
class PeriodForSystemTimeConstraint(ColumnConstraintKind):
arg_types = {"this": True, "expression": True}
class CaseSpecificColumnConstraint(ColumnConstraintKind): class CaseSpecificColumnConstraint(ColumnConstraintKind):
arg_types = {"not_": True} arg_types = {"not_": True}
@ -1351,6 +1407,10 @@ class GeneratedAsIdentityColumnConstraint(ColumnConstraintKind):
} }
class GeneratedAsRowColumnConstraint(ColumnConstraintKind):
arg_types = {"start": True, "hidden": False}
# https://dev.mysql.com/doc/refman/8.0/en/create-table.html # https://dev.mysql.com/doc/refman/8.0/en/create-table.html
class IndexColumnConstraint(ColumnConstraintKind): class IndexColumnConstraint(ColumnConstraintKind):
arg_types = { arg_types = {
@ -1383,6 +1443,11 @@ class OnUpdateColumnConstraint(ColumnConstraintKind):
pass pass
# https://docs.snowflake.com/en/sql-reference/sql/create-external-table#optional-parameters
class TransformColumnConstraint(ColumnConstraintKind):
pass
class PrimaryKeyColumnConstraint(ColumnConstraintKind): class PrimaryKeyColumnConstraint(ColumnConstraintKind):
arg_types = {"desc": False} arg_types = {"desc": False}
@ -1413,7 +1478,7 @@ class Constraint(Expression):
arg_types = {"this": True, "expressions": True} arg_types = {"this": True, "expressions": True}
class Delete(Expression): class Delete(DML):
arg_types = { arg_types = {
"with": False, "with": False,
"this": False, "this": False,
@ -1496,41 +1561,6 @@ class Delete(Expression):
**opts, **opts,
) )
def returning(
self,
expression: ExpOrStr,
dialect: DialectType = None,
copy: bool = True,
**opts,
) -> Delete:
"""
Set the RETURNING expression. Not supported by all dialects.
Example:
>>> delete("tbl").returning("*", dialect="postgres").sql()
'DELETE FROM tbl RETURNING *'
Args:
expression: the SQL code strings to parse.
If an `Expression` instance is passed, it will be used as-is.
dialect: the dialect used to parse the input expressions.
copy: if `False`, modify this expression instance in-place.
opts: other options to use to parse the input expressions.
Returns:
Delete: the modified expression.
"""
return _apply_builder(
expression=expression,
instance=self,
arg="returning",
prefix="RETURNING",
dialect=dialect,
copy=copy,
into=Returning,
**opts,
)
class Drop(Expression): class Drop(Expression):
arg_types = { arg_types = {
@ -1648,7 +1678,7 @@ class Index(Expression):
} }
class Insert(DDL): class Insert(DDL, DML):
arg_types = { arg_types = {
"with": False, "with": False,
"this": True, "this": True,
@ -2259,6 +2289,11 @@ class WithJournalTableProperty(Property):
arg_types = {"this": True} arg_types = {"this": True}
class WithSystemVersioningProperty(Property):
# this -> history table name, expression -> data consistency check
arg_types = {"this": False, "expression": False}
class Properties(Expression): class Properties(Expression):
arg_types = {"expressions": True} arg_types = {"expressions": True}
@ -3663,6 +3698,7 @@ class DataType(Expression):
Type.BIGINT, Type.BIGINT,
Type.INT128, Type.INT128,
Type.INT256, Type.INT256,
Type.BIT,
} }
FLOAT_TYPES = { FLOAT_TYPES = {
@ -3692,7 +3728,7 @@ class DataType(Expression):
@classmethod @classmethod
def build( def build(
cls, cls,
dtype: str | DataType | DataType.Type, dtype: DATA_TYPE,
dialect: DialectType = None, dialect: DialectType = None,
udt: bool = False, udt: bool = False,
**kwargs, **kwargs,
@ -3733,7 +3769,7 @@ class DataType(Expression):
return DataType(**{**data_type_exp.args, **kwargs}) return DataType(**{**data_type_exp.args, **kwargs})
def is_type(self, *dtypes: str | DataType | DataType.Type) -> bool: def is_type(self, *dtypes: DATA_TYPE) -> bool:
""" """
Checks whether this DataType matches one of the provided data types. Nested types or precision Checks whether this DataType matches one of the provided data types. Nested types or precision
will be compared using "structural equivalence" semantics, so e.g. array<int> != array<float>. will be compared using "structural equivalence" semantics, so e.g. array<int> != array<float>.
@ -3761,6 +3797,9 @@ class DataType(Expression):
return False return False
DATA_TYPE = t.Union[str, DataType, DataType.Type]
# https://www.postgresql.org/docs/15/datatype-pseudo.html # https://www.postgresql.org/docs/15/datatype-pseudo.html
class PseudoType(DataType): class PseudoType(DataType):
arg_types = {"this": True} arg_types = {"this": True}
@ -3868,7 +3907,7 @@ class BitwiseXor(Binary):
class Div(Binary): class Div(Binary):
pass arg_types = {"this": True, "expression": True, "typed": False, "safe": False}
class Overlaps(Binary): class Overlaps(Binary):
@ -3892,13 +3931,25 @@ class Dot(Binary):
return t.cast(Dot, reduce(lambda x, y: Dot(this=x, expression=y), expressions)) return t.cast(Dot, reduce(lambda x, y: Dot(this=x, expression=y), expressions))
@property
def parts(self) -> t.List[Expression]:
"""Return the parts of a table / column in order catalog, db, table."""
this, *parts = self.flatten()
parts.reverse()
for arg in ("this", "table", "db", "catalog"):
part = this.args.get(arg)
if isinstance(part, Expression):
parts.append(part)
parts.reverse()
return parts
class DPipe(Binary): class DPipe(Binary):
pass arg_types = {"this": True, "expression": True, "safe": False}
class SafeDPipe(DPipe):
pass
class EQ(Binary, Predicate): class EQ(Binary, Predicate):
@ -3913,6 +3964,11 @@ class NullSafeNEQ(Binary, Predicate):
pass pass
# Represents e.g. := in DuckDB which is mostly used for setting parameters
class PropertyEQ(Binary):
pass
class Distance(Binary): class Distance(Binary):
pass pass
@ -3981,6 +4037,11 @@ class NEQ(Binary, Predicate):
pass pass
# https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH
class Operator(Binary):
arg_types = {"this": True, "operator": True, "expression": True}
class SimilarTo(Binary, Predicate): class SimilarTo(Binary, Predicate):
pass pass
@ -4048,7 +4109,8 @@ class Between(Predicate):
class Bracket(Condition): class Bracket(Condition):
arg_types = {"this": True, "expressions": True} # https://cloud.google.com/bigquery/docs/reference/standard-sql/operators#array_subscript_operator
arg_types = {"this": True, "expressions": True, "offset": False, "safe": False}
@property @property
def output_name(self) -> str: def output_name(self) -> str:
@ -4058,10 +4120,6 @@ class Bracket(Condition):
return super().output_name return super().output_name
class SafeBracket(Bracket):
"""Represents array lookup where OOB index yields NULL instead of causing a failure."""
class Distinct(Expression): class Distinct(Expression):
arg_types = {"expressions": False, "on": False} arg_types = {"expressions": False, "on": False}
@ -4077,6 +4135,11 @@ class In(Predicate):
} }
# https://cloud.google.com/bigquery/docs/reference/standard-sql/procedural-language#for-in
class ForIn(Expression):
arg_types = {"this": True, "expression": True}
class TimeUnit(Expression): class TimeUnit(Expression):
"""Automatically converts unit arg into a var.""" """Automatically converts unit arg into a var."""
@ -4248,8 +4311,9 @@ class Array(Func):
# https://docs.snowflake.com/en/sql-reference/functions/to_char # https://docs.snowflake.com/en/sql-reference/functions/to_char
# https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/TO_CHAR-number.html
class ToChar(Func): class ToChar(Func):
arg_types = {"this": True, "format": False} arg_types = {"this": True, "format": False, "nlsparam": False}
class GenerateSeries(Func): class GenerateSeries(Func):
@ -4260,6 +4324,10 @@ class ArrayAgg(AggFunc):
pass pass
class ArrayUniqueAgg(AggFunc):
pass
class ArrayAll(Func): class ArrayAll(Func):
arg_types = {"this": True, "expression": True} arg_types = {"this": True, "expression": True}
@ -4358,7 +4426,7 @@ class Cast(Func):
def output_name(self) -> str: def output_name(self) -> str:
return self.name return self.name
def is_type(self, *dtypes: str | DataType | DataType.Type) -> bool: def is_type(self, *dtypes: DATA_TYPE) -> bool:
""" """
Checks whether this Cast's DataType matches one of the provided data types. Nested types Checks whether this Cast's DataType matches one of the provided data types. Nested types
like arrays or structs will be compared using "structural equivalence" semantics, so e.g. like arrays or structs will be compared using "structural equivalence" semantics, so e.g.
@ -4403,14 +4471,10 @@ class Chr(Func):
class Concat(Func): class Concat(Func):
arg_types = {"expressions": True} arg_types = {"expressions": True, "safe": False, "coalesce": False}
is_var_len_args = True is_var_len_args = True
class SafeConcat(Concat):
pass
class ConcatWs(Concat): class ConcatWs(Concat):
_sql_names = ["CONCAT_WS"] _sql_names = ["CONCAT_WS"]
@ -4643,6 +4707,10 @@ class If(Func):
arg_types = {"this": True, "true": True, "false": False} arg_types = {"this": True, "true": True, "false": False}
class Nullif(Func):
arg_types = {"this": True, "expression": True}
class Initcap(Func): class Initcap(Func):
arg_types = {"this": True, "expression": False} arg_types = {"this": True, "expression": False}
@ -4651,6 +4719,10 @@ class IsNan(Func):
_sql_names = ["IS_NAN", "ISNAN"] _sql_names = ["IS_NAN", "ISNAN"]
class IsInf(Func):
_sql_names = ["IS_INF", "ISINF"]
class FormatJson(Expression): class FormatJson(Expression):
pass pass
@ -4970,10 +5042,6 @@ class SafeDivide(Func):
arg_types = {"this": True, "expression": True} arg_types = {"this": True, "expression": True}
class SetAgg(AggFunc):
pass
class SHA(Func): class SHA(Func):
_sql_names = ["SHA", "SHA1"] _sql_names = ["SHA", "SHA1"]
@ -5118,6 +5186,15 @@ class Trim(Func):
class TsOrDsAdd(Func, TimeUnit): class TsOrDsAdd(Func, TimeUnit):
# return_type is used to correctly cast the arguments of this expression when transpiling it
arg_types = {"this": True, "expression": True, "unit": False, "return_type": False}
@property
def return_type(self) -> DataType:
return DataType.build(self.args.get("return_type") or DataType.Type.DATE)
class TsOrDsDiff(Func, TimeUnit):
arg_types = {"this": True, "expression": True, "unit": False} arg_types = {"this": True, "expression": True, "unit": False}
@ -5149,6 +5226,7 @@ class UnixToTime(Func):
SECONDS = Literal.string("seconds") SECONDS = Literal.string("seconds")
MILLIS = Literal.string("millis") MILLIS = Literal.string("millis")
MICROS = Literal.string("micros") MICROS = Literal.string("micros")
NANOS = Literal.string("nanos")
class UnixToTimeStr(Func): class UnixToTimeStr(Func):
@ -5202,6 +5280,7 @@ def _norm_arg(arg):
ALL_FUNCTIONS = subclasses(__name__, Func, (AggFunc, Anonymous, Func)) ALL_FUNCTIONS = subclasses(__name__, Func, (AggFunc, Anonymous, Func))
FUNCTION_BY_NAME = {name: func for func in ALL_FUNCTIONS for name in func.sql_names()}
# Helpers # Helpers
@ -5693,7 +5772,9 @@ def delete(
if where: if where:
delete_expr = delete_expr.where(where, dialect=dialect, copy=False, **opts) delete_expr = delete_expr.where(where, dialect=dialect, copy=False, **opts)
if returning: if returning:
delete_expr = delete_expr.returning(returning, dialect=dialect, copy=False, **opts) delete_expr = t.cast(
Delete, delete_expr.returning(returning, dialect=dialect, copy=False, **opts)
)
return delete_expr return delete_expr
@ -5702,6 +5783,7 @@ def insert(
into: ExpOrStr, into: ExpOrStr,
columns: t.Optional[t.Sequence[ExpOrStr]] = None, columns: t.Optional[t.Sequence[ExpOrStr]] = None,
overwrite: t.Optional[bool] = None, overwrite: t.Optional[bool] = None,
returning: t.Optional[ExpOrStr] = None,
dialect: DialectType = None, dialect: DialectType = None,
copy: bool = True, copy: bool = True,
**opts, **opts,
@ -5718,6 +5800,7 @@ def insert(
into: the tbl to insert data to. into: the tbl to insert data to.
columns: optionally the table's column names. columns: optionally the table's column names.
overwrite: whether to INSERT OVERWRITE or not. overwrite: whether to INSERT OVERWRITE or not.
returning: sql conditional parsed into a RETURNING statement
dialect: the dialect used to parse the input expressions. dialect: the dialect used to parse the input expressions.
copy: whether or not to copy the expression. copy: whether or not to copy the expression.
**opts: other options to use to parse the input expressions. **opts: other options to use to parse the input expressions.
@ -5739,7 +5822,12 @@ def insert(
**opts, **opts,
) )
return Insert(this=this, expression=expr, overwrite=overwrite) insert = Insert(this=this, expression=expr, overwrite=overwrite)
if returning:
insert = t.cast(Insert, insert.returning(returning, dialect=dialect, copy=False, **opts))
return insert
def condition( def condition(
@ -5913,7 +6001,7 @@ def to_identifier(name, quoted=None, copy=True):
return identifier return identifier
def parse_identifier(name: str, dialect: DialectType = None) -> Identifier: def parse_identifier(name: str | Identifier, dialect: DialectType = None) -> Identifier:
""" """
Parses a given string into an identifier. Parses a given string into an identifier.
@ -5965,7 +6053,7 @@ def to_table(sql_path: None, **kwargs) -> None:
def to_table( def to_table(
sql_path: t.Optional[str | Table], dialect: DialectType = None, **kwargs sql_path: t.Optional[str | Table], dialect: DialectType = None, copy: bool = True, **kwargs
) -> t.Optional[Table]: ) -> t.Optional[Table]:
""" """
Create a table expression from a `[catalog].[schema].[table]` sql path. Catalog and schema are optional. Create a table expression from a `[catalog].[schema].[table]` sql path. Catalog and schema are optional.
@ -5974,13 +6062,14 @@ def to_table(
Args: Args:
sql_path: a `[catalog].[schema].[table]` string. sql_path: a `[catalog].[schema].[table]` string.
dialect: the source dialect according to which the table name will be parsed. dialect: the source dialect according to which the table name will be parsed.
copy: Whether or not to copy a table if it is passed in.
kwargs: the kwargs to instantiate the resulting `Table` expression with. kwargs: the kwargs to instantiate the resulting `Table` expression with.
Returns: Returns:
A table expression. A table expression.
""" """
if sql_path is None or isinstance(sql_path, Table): if sql_path is None or isinstance(sql_path, Table):
return sql_path return maybe_copy(sql_path, copy=copy)
if not isinstance(sql_path, str): if not isinstance(sql_path, str):
raise ValueError(f"Invalid type provided for a table: {type(sql_path)}") raise ValueError(f"Invalid type provided for a table: {type(sql_path)}")
@ -6123,7 +6212,7 @@ def column(
) )
def cast(expression: ExpOrStr, to: str | DataType | DataType.Type, **opts) -> Cast: def cast(expression: ExpOrStr, to: DATA_TYPE, **opts) -> Cast:
"""Cast an expression to a data type. """Cast an expression to a data type.
Example: Example:
@ -6335,12 +6424,15 @@ def column_table_names(expression: Expression, exclude: str = "") -> t.Set[str]:
} }
def table_name(table: Table | str, dialect: DialectType = None) -> str: def table_name(table: Table | str, dialect: DialectType = None, identify: bool = False) -> str:
"""Get the full name of a table as a string. """Get the full name of a table as a string.
Args: Args:
table: Table expression node or string. table: Table expression node or string.
dialect: The dialect to generate the table name for. dialect: The dialect to generate the table name for.
identify: Determines when an identifier should be quoted. Possible values are:
False (default): Never quote, except in cases where it's mandatory by the dialect.
True: Always quote.
Examples: Examples:
>>> from sqlglot import exp, parse_one >>> from sqlglot import exp, parse_one
@ -6358,37 +6450,68 @@ def table_name(table: Table | str, dialect: DialectType = None) -> str:
return ".".join( return ".".join(
part.sql(dialect=dialect, identify=True) part.sql(dialect=dialect, identify=True)
if not SAFE_IDENTIFIER_RE.match(part.name) if identify or not SAFE_IDENTIFIER_RE.match(part.name)
else part.name else part.name
for part in table.parts for part in table.parts
) )
def replace_tables(expression: E, mapping: t.Dict[str, str], copy: bool = True) -> E: def normalize_table_name(table: str | Table, dialect: DialectType = None, copy: bool = True) -> str:
"""Returns a case normalized table name without quotes.
Args:
table: the table to normalize
dialect: the dialect to use for normalization rules
copy: whether or not to copy the expression.
Examples:
>>> normalize_table_name("`A-B`.c", dialect="bigquery")
'A-B.c'
"""
from sqlglot.optimizer.normalize_identifiers import normalize_identifiers
return ".".join(
p.name
for p in normalize_identifiers(
to_table(table, dialect=dialect, copy=copy), dialect=dialect
).parts
)
def replace_tables(
expression: E, mapping: t.Dict[str, str], dialect: DialectType = None, copy: bool = True
) -> E:
"""Replace all tables in expression according to the mapping. """Replace all tables in expression according to the mapping.
Args: Args:
expression: expression node to be transformed and replaced. expression: expression node to be transformed and replaced.
mapping: mapping of table names. mapping: mapping of table names.
dialect: the dialect of the mapping table
copy: whether or not to copy the expression. copy: whether or not to copy the expression.
Examples: Examples:
>>> from sqlglot import exp, parse_one >>> from sqlglot import exp, parse_one
>>> replace_tables(parse_one("select * from a.b"), {"a.b": "c"}).sql() >>> replace_tables(parse_one("select * from a.b"), {"a.b": "c"}).sql()
'SELECT * FROM c' 'SELECT * FROM c /* a.b */'
Returns: Returns:
The mapped expression. The mapped expression.
""" """
mapping = {normalize_table_name(k, dialect=dialect): v for k, v in mapping.items()}
def _replace_tables(node: Expression) -> Expression: def _replace_tables(node: Expression) -> Expression:
if isinstance(node, Table): if isinstance(node, Table):
new_name = mapping.get(table_name(node)) original = normalize_table_name(node, dialect=dialect)
new_name = mapping.get(original)
if new_name: if new_name:
return to_table( table = to_table(
new_name, new_name,
**{k: v for k, v in node.args.items() if k not in ("this", "db", "catalog")}, **{k: v for k, v in node.args.items() if k not in TABLE_PARTS},
) )
table.add_comments([original])
return table
return node return node
return expression.transform(_replace_tables, copy=copy) return expression.transform(_replace_tables, copy=copy)
@ -6431,7 +6554,10 @@ def replace_placeholders(expression: Expression, *args, **kwargs) -> Expression:
def expand( def expand(
expression: Expression, sources: t.Dict[str, Subqueryable], copy: bool = True expression: Expression,
sources: t.Dict[str, Subqueryable],
dialect: DialectType = None,
copy: bool = True,
) -> Expression: ) -> Expression:
"""Transforms an expression by expanding all referenced sources into subqueries. """Transforms an expression by expanding all referenced sources into subqueries.
@ -6446,15 +6572,17 @@ def expand(
Args: Args:
expression: The expression to expand. expression: The expression to expand.
sources: A dictionary of name to Subqueryables. sources: A dictionary of name to Subqueryables.
dialect: The dialect of the sources dict.
copy: Whether or not to copy the expression during transformation. Defaults to True. copy: Whether or not to copy the expression during transformation. Defaults to True.
Returns: Returns:
The transformed expression. The transformed expression.
""" """
sources = {normalize_table_name(k, dialect=dialect): v for k, v in sources.items()}
def _expand(node: Expression): def _expand(node: Expression):
if isinstance(node, Table): if isinstance(node, Table):
name = table_name(node) name = normalize_table_name(node, dialect=dialect)
source = sources.get(name) source = sources.get(name)
if source: if source:
subquery = source.subquery(node.alias or name) subquery = source.subquery(node.alias or name)
@ -6465,7 +6593,7 @@ def expand(
return expression.transform(_expand, copy=copy) return expression.transform(_expand, copy=copy)
def func(name: str, *args, dialect: DialectType = None, **kwargs) -> Func: def func(name: str, *args, copy: bool = True, dialect: DialectType = None, **kwargs) -> Func:
""" """
Returns a Func expression. Returns a Func expression.
@ -6479,6 +6607,7 @@ def func(name: str, *args, dialect: DialectType = None, **kwargs) -> Func:
Args: Args:
name: the name of the function to build. name: the name of the function to build.
args: the args used to instantiate the function of interest. args: the args used to instantiate the function of interest.
copy: whether or not to copy the argument expressions.
dialect: the source dialect. dialect: the source dialect.
kwargs: the kwargs used to instantiate the function of interest. kwargs: the kwargs used to instantiate the function of interest.
@ -6494,14 +6623,29 @@ def func(name: str, *args, dialect: DialectType = None, **kwargs) -> Func:
from sqlglot.dialects.dialect import Dialect from sqlglot.dialects.dialect import Dialect
converted: t.List[Expression] = [maybe_parse(arg, dialect=dialect) for arg in args] dialect = Dialect.get_or_raise(dialect)
kwargs = {key: maybe_parse(value, dialect=dialect) for key, value in kwargs.items()}
parser = Dialect.get_or_raise(dialect)().parser() converted: t.List[Expression] = [maybe_parse(arg, dialect=dialect, copy=copy) for arg in args]
from_args_list = parser.FUNCTIONS.get(name.upper()) kwargs = {key: maybe_parse(value, dialect=dialect, copy=copy) for key, value in kwargs.items()}
if from_args_list: constructor = dialect.parser_class.FUNCTIONS.get(name.upper())
function = from_args_list(converted) if converted else from_args_list.__self__(**kwargs) # type: ignore if constructor:
if converted:
if "dialect" in constructor.__code__.co_varnames:
function = constructor(converted, dialect=dialect)
else:
function = constructor(converted)
elif constructor.__name__ == "from_arg_list":
function = constructor.__self__(**kwargs) # type: ignore
else:
constructor = FUNCTION_BY_NAME.get(name.upper())
if constructor:
function = constructor(**kwargs)
else:
raise ValueError(
f"Unable to convert '{name}' into a Func. Either manually construct "
"the Func expression of interest or parse the function call."
)
else: else:
kwargs = kwargs or {"expressions": converted} kwargs = kwargs or {"expressions": converted}
function = Anonymous(this=name, **kwargs) function = Anonymous(this=name, **kwargs)
@ -6512,6 +6656,48 @@ def func(name: str, *args, dialect: DialectType = None, **kwargs) -> Func:
return function return function
def case(
expression: t.Optional[ExpOrStr] = None,
**opts,
) -> Case:
"""
Initialize a CASE statement.
Example:
case().when("a = 1", "foo").else_("bar")
Args:
expression: Optionally, the input expression (not all dialects support this)
**opts: Extra keyword arguments for parsing `expression`
"""
if expression is not None:
this = maybe_parse(expression, **opts)
else:
this = None
return Case(this=this, ifs=[])
def cast_unless(
expression: ExpOrStr,
to: DATA_TYPE,
*types: DATA_TYPE,
**opts: t.Any,
) -> Expression | Cast:
"""
Cast an expression to a data type unless it is a specified type.
Args:
expression: The expression to cast.
to: The data type to cast to.
**types: The types to exclude from casting.
**opts: Extra keyword arguments for parsing `expression`
"""
expr = maybe_parse(expression, **opts)
if expr.is_type(*types):
return expr
return cast(expr, to, **opts)
def true() -> Boolean: def true() -> Boolean:
""" """
Returns a true Boolean expression. Returns a true Boolean expression.

344
sqlglot/generator.py
View file

@ -9,10 +9,11 @@ from sqlglot import exp
from sqlglot.errors import ErrorLevel, UnsupportedError, concat_messages from sqlglot.errors import ErrorLevel, UnsupportedError, concat_messages
from sqlglot.helper import apply_index_offset, csv, seq_get from sqlglot.helper import apply_index_offset, csv, seq_get
from sqlglot.time import format_time from sqlglot.time import format_time
from sqlglot.tokens import Tokenizer, TokenType from sqlglot.tokens import TokenType
if t.TYPE_CHECKING: if t.TYPE_CHECKING:
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.dialects.dialect import DialectType
logger = logging.getLogger("sqlglot") logger = logging.getLogger("sqlglot")
@ -58,9 +59,6 @@ class Generator:
exp.DateAdd: lambda self, e: self.func( exp.DateAdd: lambda self, e: self.func(
"DATE_ADD", e.this, e.expression, exp.Literal.string(e.text("unit")) "DATE_ADD", e.this, e.expression, exp.Literal.string(e.text("unit"))
), ),
exp.TsOrDsAdd: lambda self, e: self.func(
"TS_OR_DS_ADD", e.this, e.expression, exp.Literal.string(e.text("unit"))
),
exp.CaseSpecificColumnConstraint: lambda self, e: f"{'NOT ' if e.args.get('not_') else ''}CASESPECIFIC", exp.CaseSpecificColumnConstraint: lambda self, e: f"{'NOT ' if e.args.get('not_') else ''}CASESPECIFIC",
exp.CharacterSetColumnConstraint: lambda self, e: f"CHARACTER SET {self.sql(e, 'this')}", exp.CharacterSetColumnConstraint: lambda self, e: f"CHARACTER SET {self.sql(e, 'this')}",
exp.CharacterSetProperty: lambda self, e: f"{'DEFAULT ' if e.args.get('default') else ''}CHARACTER SET={self.sql(e, 'this')}", exp.CharacterSetProperty: lambda self, e: f"{'DEFAULT ' if e.args.get('default') else ''}CHARACTER SET={self.sql(e, 'this')}",
@ -108,9 +106,6 @@ class Generator:
exp.WithJournalTableProperty: lambda self, e: f"WITH JOURNAL TABLE={self.sql(e, 'this')}", exp.WithJournalTableProperty: lambda self, e: f"WITH JOURNAL TABLE={self.sql(e, 'this')}",
} }
# Whether the base comes first
LOG_BASE_FIRST = True
# Whether or not null ordering is supported in order by # Whether or not null ordering is supported in order by
NULL_ORDERING_SUPPORTED = True NULL_ORDERING_SUPPORTED = True
@ -201,7 +196,7 @@ class Generator:
VALUES_AS_TABLE = True VALUES_AS_TABLE = True
# Whether or not the word COLUMN is included when adding a column with ALTER TABLE # Whether or not the word COLUMN is included when adding a column with ALTER TABLE
ALTER_TABLE_ADD_COLUMN_KEYWORD = True ALTER_TABLE_INCLUDE_COLUMN_KEYWORD = True
# UNNEST WITH ORDINALITY (presto) instead of UNNEST WITH OFFSET (bigquery) # UNNEST WITH ORDINALITY (presto) instead of UNNEST WITH OFFSET (bigquery)
UNNEST_WITH_ORDINALITY = True UNNEST_WITH_ORDINALITY = True
@ -212,9 +207,6 @@ class Generator:
# Whether or not JOIN sides (LEFT, RIGHT) are supported in conjunction with SEMI/ANTI join kinds # Whether or not JOIN sides (LEFT, RIGHT) are supported in conjunction with SEMI/ANTI join kinds
SEMI_ANTI_JOIN_WITH_SIDE = True SEMI_ANTI_JOIN_WITH_SIDE = True
# Whether or not session variables / parameters are supported, e.g. @x in T-SQL
SUPPORTS_PARAMETERS = True
# Whether or not to include the type of a computed column in the CREATE DDL # Whether or not to include the type of a computed column in the CREATE DDL
COMPUTED_COLUMN_WITH_TYPE = True COMPUTED_COLUMN_WITH_TYPE = True
@ -230,12 +222,15 @@ class Generator:
# Whether or not data types support additional specifiers like e.g. CHAR or BYTE (oracle) # Whether or not data types support additional specifiers like e.g. CHAR or BYTE (oracle)
DATA_TYPE_SPECIFIERS_ALLOWED = False DATA_TYPE_SPECIFIERS_ALLOWED = False
# Whether or not nested CTEs (e.g. defined inside of subqueries) are allowed # Whether or not conditions require booleans WHERE x = 0 vs WHERE x
SUPPORTS_NESTED_CTES = True ENSURE_BOOLS = False
# Whether or not the "RECURSIVE" keyword is required when defining recursive CTEs # Whether or not the "RECURSIVE" keyword is required when defining recursive CTEs
CTE_RECURSIVE_KEYWORD_REQUIRED = True CTE_RECURSIVE_KEYWORD_REQUIRED = True
# Whether or not CONCAT requires >1 arguments
SUPPORTS_SINGLE_ARG_CONCAT = True
TYPE_MAPPING = { TYPE_MAPPING = {
exp.DataType.Type.NCHAR: "CHAR", exp.DataType.Type.NCHAR: "CHAR",
exp.DataType.Type.NVARCHAR: "VARCHAR", exp.DataType.Type.NVARCHAR: "VARCHAR",
@ -335,6 +330,7 @@ class Generator:
exp.VolatileProperty: exp.Properties.Location.POST_CREATE, exp.VolatileProperty: exp.Properties.Location.POST_CREATE,
exp.WithDataProperty: exp.Properties.Location.POST_EXPRESSION, exp.WithDataProperty: exp.Properties.Location.POST_EXPRESSION,
exp.WithJournalTableProperty: exp.Properties.Location.POST_NAME, exp.WithJournalTableProperty: exp.Properties.Location.POST_NAME,
exp.WithSystemVersioningProperty: exp.Properties.Location.POST_SCHEMA,
} }
# Keywords that can't be used as unquoted identifier names # Keywords that can't be used as unquoted identifier names
@ -368,37 +364,13 @@ class Generator:
exp.Paren, exp.Paren,
) )
# Expressions that need to have all CTEs under them bubbled up to them
EXPRESSIONS_WITHOUT_NESTED_CTES: t.Set[t.Type[exp.Expression]] = set()
KEY_VALUE_DEFINITONS = (exp.Bracket, exp.EQ, exp.PropertyEQ, exp.Slice)
SENTINEL_LINE_BREAK = "__SQLGLOT__LB__" SENTINEL_LINE_BREAK = "__SQLGLOT__LB__"
# Autofilled
INVERSE_TIME_MAPPING: t.Dict[str, str] = {}
INVERSE_TIME_TRIE: t.Dict = {}
INVERSE_ESCAPE_SEQUENCES: t.Dict[str, str] = {}
INDEX_OFFSET = 0
UNNEST_COLUMN_ONLY = False
ALIAS_POST_TABLESAMPLE = False
IDENTIFIERS_CAN_START_WITH_DIGIT = False
STRICT_STRING_CONCAT = False
NORMALIZE_FUNCTIONS: bool | str = "upper"
NULL_ORDERING = "nulls_are_small"
can_identify: t.Callable[[str, str | bool], bool]
# Delimiters for quotes, identifiers and the corresponding escape characters
QUOTE_START = "'"
QUOTE_END = "'"
IDENTIFIER_START = '"'
IDENTIFIER_END = '"'
TOKENIZER_CLASS = Tokenizer
# Delimiters for bit, hex, byte and raw literals
BIT_START: t.Optional[str] = None
BIT_END: t.Optional[str] = None
HEX_START: t.Optional[str] = None
HEX_END: t.Optional[str] = None
BYTE_START: t.Optional[str] = None
BYTE_END: t.Optional[str] = None
__slots__ = ( __slots__ = (
"pretty", "pretty",
"identify", "identify",
@ -411,6 +383,7 @@ class Generator:
"leading_comma", "leading_comma",
"max_text_width", "max_text_width",
"comments", "comments",
"dialect",
"unsupported_messages", "unsupported_messages",
"_escaped_quote_end", "_escaped_quote_end",
"_escaped_identifier_end", "_escaped_identifier_end",
@ -429,8 +402,10 @@ class Generator:
leading_comma: bool = False, leading_comma: bool = False,
max_text_width: int = 80, max_text_width: int = 80,
comments: bool = True, comments: bool = True,
dialect: DialectType = None,
): ):
import sqlglot import sqlglot
from sqlglot.dialects import Dialect
self.pretty = pretty if pretty is not None else sqlglot.pretty self.pretty = pretty if pretty is not None else sqlglot.pretty
self.identify = identify self.identify = identify
@ -442,16 +417,19 @@ class Generator:
self.leading_comma = leading_comma self.leading_comma = leading_comma
self.max_text_width = max_text_width self.max_text_width = max_text_width
self.comments = comments self.comments = comments
self.dialect = Dialect.get_or_raise(dialect)
# This is both a Dialect property and a Generator argument, so we prioritize the latter # This is both a Dialect property and a Generator argument, so we prioritize the latter
self.normalize_functions = ( self.normalize_functions = (
self.NORMALIZE_FUNCTIONS if normalize_functions is None else normalize_functions self.dialect.NORMALIZE_FUNCTIONS if normalize_functions is None else normalize_functions
) )
self.unsupported_messages: t.List[str] = [] self.unsupported_messages: t.List[str] = []
self._escaped_quote_end: str = self.TOKENIZER_CLASS.STRING_ESCAPES[0] + self.QUOTE_END self._escaped_quote_end: str = (
self.dialect.tokenizer_class.STRING_ESCAPES[0] + self.dialect.QUOTE_END
)
self._escaped_identifier_end: str = ( self._escaped_identifier_end: str = (
self.TOKENIZER_CLASS.IDENTIFIER_ESCAPES[0] + self.IDENTIFIER_END self.dialect.tokenizer_class.IDENTIFIER_ESCAPES[0] + self.dialect.IDENTIFIER_END
) )
def generate(self, expression: exp.Expression, copy: bool = True) -> str: def generate(self, expression: exp.Expression, copy: bool = True) -> str:
@ -469,23 +447,14 @@ class Generator:
if copy: if copy:
expression = expression.copy() expression = expression.copy()
# Some dialects only support CTEs at the top level expression, so we need to bubble up nested expression = self.preprocess(expression)
# CTEs to that level in order to produce a syntactically valid expression. This transformation
# happens here to minimize code duplication, since many expressions support CTEs.
if (
not self.SUPPORTS_NESTED_CTES
and isinstance(expression, exp.Expression)
and not expression.parent
and "with" in expression.arg_types
and any(node.parent is not expression for node in expression.find_all(exp.With))
):
from sqlglot.transforms import move_ctes_to_top_level
expression = move_ctes_to_top_level(expression)
self.unsupported_messages = [] self.unsupported_messages = []
sql = self.sql(expression).strip() sql = self.sql(expression).strip()
if self.pretty:
sql = sql.replace(self.SENTINEL_LINE_BREAK, "\n")
if self.unsupported_level == ErrorLevel.IGNORE: if self.unsupported_level == ErrorLevel.IGNORE:
return sql return sql
@ -495,10 +464,26 @@ class Generator:
elif self.unsupported_level == ErrorLevel.RAISE and self.unsupported_messages: elif self.unsupported_level == ErrorLevel.RAISE and self.unsupported_messages:
raise UnsupportedError(concat_messages(self.unsupported_messages, self.max_unsupported)) raise UnsupportedError(concat_messages(self.unsupported_messages, self.max_unsupported))
if self.pretty:
sql = sql.replace(self.SENTINEL_LINE_BREAK, "\n")
return sql return sql
def preprocess(self, expression: exp.Expression) -> exp.Expression:
"""Apply generic preprocessing transformations to a given expression."""
if (
not expression.parent
and type(expression) in self.EXPRESSIONS_WITHOUT_NESTED_CTES
and any(node.parent is not expression for node in expression.find_all(exp.With))
):
from sqlglot.transforms import move_ctes_to_top_level
expression = move_ctes_to_top_level(expression)
if self.ENSURE_BOOLS:
from sqlglot.transforms import ensure_bools
expression = ensure_bools(expression)
return expression
def unsupported(self, message: str) -> None: def unsupported(self, message: str) -> None:
if self.unsupported_level == ErrorLevel.IMMEDIATE: if self.unsupported_level == ErrorLevel.IMMEDIATE:
raise UnsupportedError(message) raise UnsupportedError(message)
@ -752,9 +737,24 @@ class Generator:
return f"GENERATED{this} AS {expr}{sequence_opts}" return f"GENERATED{this} AS {expr}{sequence_opts}"
def generatedasrowcolumnconstraint_sql(
self, expression: exp.GeneratedAsRowColumnConstraint
) -> str:
start = "START" if expression.args["start"] else "END"
hidden = " HIDDEN" if expression.args.get("hidden") else ""
return f"GENERATED ALWAYS AS ROW {start}{hidden}"
def periodforsystemtimeconstraint_sql(
self, expression: exp.PeriodForSystemTimeConstraint
) -> str:
return f"PERIOD FOR SYSTEM_TIME ({self.sql(expression, 'this')}, {self.sql(expression, 'expression')})"
def notnullcolumnconstraint_sql(self, expression: exp.NotNullColumnConstraint) -> str: def notnullcolumnconstraint_sql(self, expression: exp.NotNullColumnConstraint) -> str:
return f"{'' if expression.args.get('allow_null') else 'NOT '}NULL" return f"{'' if expression.args.get('allow_null') else 'NOT '}NULL"
def transformcolumnconstraint_sql(self, expression: exp.TransformColumnConstraint) -> str:
return f"AS {self.sql(expression, 'this')}"
def primarykeycolumnconstraint_sql(self, expression: exp.PrimaryKeyColumnConstraint) -> str: def primarykeycolumnconstraint_sql(self, expression: exp.PrimaryKeyColumnConstraint) -> str:
desc = expression.args.get("desc") desc = expression.args.get("desc")
if desc is not None: if desc is not None:
@ -900,32 +900,32 @@ class Generator:
columns = self.expressions(expression, key="columns", flat=True) columns = self.expressions(expression, key="columns", flat=True)
columns = f"({columns})" if columns else "" columns = f"({columns})" if columns else ""
if not alias and not self.UNNEST_COLUMN_ONLY: if not alias and not self.dialect.UNNEST_COLUMN_ONLY:
alias = "_t" alias = "_t"
return f"{alias}{columns}" return f"{alias}{columns}"
def bitstring_sql(self, expression: exp.BitString) -> str: def bitstring_sql(self, expression: exp.BitString) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
if self.BIT_START: if self.dialect.BIT_START:
return f"{self.BIT_START}{this}{self.BIT_END}" return f"{self.dialect.BIT_START}{this}{self.dialect.BIT_END}"
return f"{int(this, 2)}" return f"{int(this, 2)}"
def hexstring_sql(self, expression: exp.HexString) -> str: def hexstring_sql(self, expression: exp.HexString) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
if self.HEX_START: if self.dialect.HEX_START:
return f"{self.HEX_START}{this}{self.HEX_END}" return f"{self.dialect.HEX_START}{this}{self.dialect.HEX_END}"
return f"{int(this, 16)}" return f"{int(this, 16)}"
def bytestring_sql(self, expression: exp.ByteString) -> str: def bytestring_sql(self, expression: exp.ByteString) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
if self.BYTE_START: if self.dialect.BYTE_START:
return f"{self.BYTE_START}{this}{self.BYTE_END}" return f"{self.dialect.BYTE_START}{this}{self.dialect.BYTE_END}"
return this return this
def rawstring_sql(self, expression: exp.RawString) -> str: def rawstring_sql(self, expression: exp.RawString) -> str:
string = self.escape_str(expression.this.replace("\\", "\\\\")) string = self.escape_str(expression.this.replace("\\", "\\\\"))
return f"{self.QUOTE_START}{string}{self.QUOTE_END}" return f"{self.dialect.QUOTE_START}{string}{self.dialect.QUOTE_END}"
def datatypeparam_sql(self, expression: exp.DataTypeParam) -> str: def datatypeparam_sql(self, expression: exp.DataTypeParam) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
@ -1065,14 +1065,14 @@ class Generator:
text = expression.name text = expression.name
lower = text.lower() lower = text.lower()
text = lower if self.normalize and not expression.quoted else text text = lower if self.normalize and not expression.quoted else text
text = text.replace(self.IDENTIFIER_END, self._escaped_identifier_end) text = text.replace(self.dialect.IDENTIFIER_END, self._escaped_identifier_end)
if ( if (
expression.quoted expression.quoted
or self.can_identify(text, self.identify) or self.dialect.can_identify(text, self.identify)
or lower in self.RESERVED_KEYWORDS or lower in self.RESERVED_KEYWORDS
or (not self.IDENTIFIERS_CAN_START_WITH_DIGIT and text[:1].isdigit()) or (not self.dialect.IDENTIFIERS_CAN_START_WITH_DIGIT and text[:1].isdigit())
): ):
text = f"{self.IDENTIFIER_START}{text}{self.IDENTIFIER_END}" text = f"{self.dialect.IDENTIFIER_START}{text}{self.dialect.IDENTIFIER_END}"
return text return text
def inputoutputformat_sql(self, expression: exp.InputOutputFormat) -> str: def inputoutputformat_sql(self, expression: exp.InputOutputFormat) -> str:
@ -1121,7 +1121,7 @@ class Generator:
expressions = self.expressions(properties, sep=sep, indent=False) expressions = self.expressions(properties, sep=sep, indent=False)
if expressions: if expressions:
expressions = self.wrap(expressions) if wrapped else expressions expressions = self.wrap(expressions) if wrapped else expressions
return f"{prefix}{' ' if prefix and prefix != ' ' else ''}{expressions}{suffix}" return f"{prefix}{' ' if prefix.strip() else ''}{expressions}{suffix}"
return "" return ""
def with_properties(self, properties: exp.Properties) -> str: def with_properties(self, properties: exp.Properties) -> str:
@ -1286,6 +1286,21 @@ class Generator:
statistics_sql = f" AND {'NO ' if not statistics else ''}STATISTICS" statistics_sql = f" AND {'NO ' if not statistics else ''}STATISTICS"
return f"{data_sql}{statistics_sql}" return f"{data_sql}{statistics_sql}"
def withsystemversioningproperty_sql(self, expression: exp.WithSystemVersioningProperty) -> str:
sql = "WITH(SYSTEM_VERSIONING=ON"
if expression.this:
history_table = self.sql(expression, "this")
sql = f"{sql}(HISTORY_TABLE={history_table}"
if expression.expression:
data_consistency_check = self.sql(expression, "expression")
sql = f"{sql}, DATA_CONSISTENCY_CHECK={data_consistency_check}"
sql = f"{sql})"
return f"{sql})"
def insert_sql(self, expression: exp.Insert) -> str: def insert_sql(self, expression: exp.Insert) -> str:
overwrite = expression.args.get("overwrite") overwrite = expression.args.get("overwrite")
@ -1387,13 +1402,13 @@ class Generator:
def table_sql(self, expression: exp.Table, sep: str = " AS ") -> str: def table_sql(self, expression: exp.Table, sep: str = " AS ") -> str:
table = ".".join( table = ".".join(
part self.sql(part)
for part in [ for part in (
self.sql(expression, "catalog"), expression.args.get("catalog"),
self.sql(expression, "db"), expression.args.get("db"),
self.sql(expression, "this"), expression.args.get("this"),
] )
if part if part is not None
) )
version = self.sql(expression, "version") version = self.sql(expression, "version")
@ -1426,7 +1441,7 @@ class Generator:
def tablesample_sql( def tablesample_sql(
self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS " self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS "
) -> str: ) -> str:
if self.ALIAS_POST_TABLESAMPLE and expression.this.alias: if self.dialect.ALIAS_POST_TABLESAMPLE and expression.this and expression.this.alias:
table = expression.this.copy() table = expression.this.copy()
table.set("alias", None) table.set("alias", None)
this = self.sql(table) this = self.sql(table)
@ -1676,12 +1691,16 @@ class Generator:
def limit_sql(self, expression: exp.Limit, top: bool = False) -> str: def limit_sql(self, expression: exp.Limit, top: bool = False) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
args = ", ".join(
self.sql(self._simplify_unless_literal(e) if self.LIMIT_ONLY_LITERALS else e) args = [
self._simplify_unless_literal(e) if self.LIMIT_ONLY_LITERALS else e
for e in (expression.args.get(k) for k in ("offset", "expression")) for e in (expression.args.get(k) for k in ("offset", "expression"))
if e if e
) ]
return f"{this}{self.seg('TOP' if top else 'LIMIT')} {args}"
args_sql = ", ".join(self.sql(e) for e in args)
args_sql = f"({args_sql})" if any(top and not e.is_number for e in args) else args_sql
return f"{this}{self.seg('TOP' if top else 'LIMIT')} {args_sql}"
def offset_sql(self, expression: exp.Offset) -> str: def offset_sql(self, expression: exp.Offset) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
@ -1732,13 +1751,13 @@ class Generator:
def literal_sql(self, expression: exp.Literal) -> str: def literal_sql(self, expression: exp.Literal) -> str:
text = expression.this or "" text = expression.this or ""
if expression.is_string: if expression.is_string:
text = f"{self.QUOTE_START}{self.escape_str(text)}{self.QUOTE_END}" text = f"{self.dialect.QUOTE_START}{self.escape_str(text)}{self.dialect.QUOTE_END}"
return text return text
def escape_str(self, text: str) -> str: def escape_str(self, text: str) -> str:
text = text.replace(self.QUOTE_END, self._escaped_quote_end) text = text.replace(self.dialect.QUOTE_END, self._escaped_quote_end)
if self.INVERSE_ESCAPE_SEQUENCES: if self.dialect.INVERSE_ESCAPE_SEQUENCES:
text = "".join(self.INVERSE_ESCAPE_SEQUENCES.get(ch, ch) for ch in text) text = "".join(self.dialect.INVERSE_ESCAPE_SEQUENCES.get(ch, ch) for ch in text)
elif self.pretty: elif self.pretty:
text = text.replace("\n", self.SENTINEL_LINE_BREAK) text = text.replace("\n", self.SENTINEL_LINE_BREAK)
return text return text
@ -1782,9 +1801,11 @@ class Generator:
nulls_first = expression.args.get("nulls_first") nulls_first = expression.args.get("nulls_first")
nulls_last = not nulls_first nulls_last = not nulls_first
nulls_are_large = self.NULL_ORDERING == "nulls_are_large" nulls_are_large = self.dialect.NULL_ORDERING == "nulls_are_large"
nulls_are_small = self.NULL_ORDERING == "nulls_are_small" nulls_are_small = self.dialect.NULL_ORDERING == "nulls_are_small"
nulls_are_last = self.NULL_ORDERING == "nulls_are_last" nulls_are_last = self.dialect.NULL_ORDERING == "nulls_are_last"
this = self.sql(expression, "this")
sort_order = " DESC" if desc else (" ASC" if desc is False else "") sort_order = " DESC" if desc else (" ASC" if desc is False else "")
nulls_sort_change = "" nulls_sort_change = ""
@ -1799,13 +1820,13 @@ class Generator:
): ):
nulls_sort_change = " NULLS LAST" nulls_sort_change = " NULLS LAST"
# If the NULLS FIRST/LAST clause is unsupported, we add another sort key to simulate it
if nulls_sort_change and not self.NULL_ORDERING_SUPPORTED: if nulls_sort_change and not self.NULL_ORDERING_SUPPORTED:
self.unsupported( null_sort_order = " DESC" if nulls_sort_change == " NULLS FIRST" else ""
"Sorting in an ORDER BY on NULLS FIRST/NULLS LAST is not supported by this dialect" this = f"CASE WHEN {this} IS NULL THEN 1 ELSE 0 END{null_sort_order}, {this}"
)
nulls_sort_change = "" nulls_sort_change = ""
return f"{self.sql(expression, 'this')}{sort_order}{nulls_sort_change}" return f"{this}{sort_order}{nulls_sort_change}"
def matchrecognize_sql(self, expression: exp.MatchRecognize) -> str: def matchrecognize_sql(self, expression: exp.MatchRecognize) -> str:
partition = self.partition_by_sql(expression) partition = self.partition_by_sql(expression)
@ -1933,10 +1954,13 @@ class Generator:
) )
kind = "" kind = ""
# We use LIMIT_IS_TOP as a proxy for whether DISTINCT should go first because tsql and Teradata
# are the only dialects that use LIMIT_IS_TOP and both place DISTINCT first.
top_distinct = f"{distinct}{hint}{top}" if self.LIMIT_IS_TOP else f"{top}{hint}{distinct}"
expressions = f"{self.sep()}{expressions}" if expressions else expressions expressions = f"{self.sep()}{expressions}" if expressions else expressions
sql = self.query_modifiers( sql = self.query_modifiers(
expression, expression,
f"SELECT{top}{hint}{distinct}{kind}{expressions}", f"SELECT{top_distinct}{kind}{expressions}",
self.sql(expression, "into", comment=False), self.sql(expression, "into", comment=False),
self.sql(expression, "from", comment=False), self.sql(expression, "from", comment=False),
) )
@ -1961,7 +1985,7 @@ class Generator:
def parameter_sql(self, expression: exp.Parameter) -> str: def parameter_sql(self, expression: exp.Parameter) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
return f"{self.PARAMETER_TOKEN}{this}" if self.SUPPORTS_PARAMETERS else this return f"{self.PARAMETER_TOKEN}{this}"
def sessionparameter_sql(self, expression: exp.SessionParameter) -> str: def sessionparameter_sql(self, expression: exp.SessionParameter) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
@ -2009,7 +2033,7 @@ class Generator:
if alias and isinstance(offset, exp.Expression): if alias and isinstance(offset, exp.Expression):
alias.append("columns", offset) alias.append("columns", offset)
if alias and self.UNNEST_COLUMN_ONLY: if alias and self.dialect.UNNEST_COLUMN_ONLY:
columns = alias.columns columns = alias.columns
alias = self.sql(columns[0]) if columns else "" alias = self.sql(columns[0]) if columns else ""
else: else:
@ -2080,14 +2104,14 @@ class Generator:
return f"{this} BETWEEN {low} AND {high}" return f"{this} BETWEEN {low} AND {high}"
def bracket_sql(self, expression: exp.Bracket) -> str: def bracket_sql(self, expression: exp.Bracket) -> str:
expressions = apply_index_offset(expression.this, expression.expressions, self.INDEX_OFFSET) expressions = apply_index_offset(
expression.this,
expression.expressions,
self.dialect.INDEX_OFFSET - expression.args.get("offset", 0),
)
expressions_sql = ", ".join(self.sql(e) for e in expressions) expressions_sql = ", ".join(self.sql(e) for e in expressions)
return f"{self.sql(expression, 'this')}[{expressions_sql}]" return f"{self.sql(expression, 'this')}[{expressions_sql}]"
def safebracket_sql(self, expression: exp.SafeBracket) -> str:
return self.bracket_sql(expression)
def all_sql(self, expression: exp.All) -> str: def all_sql(self, expression: exp.All) -> str:
return f"ALL {self.wrap(expression)}" return f"ALL {self.wrap(expression)}"
@ -2145,12 +2169,33 @@ class Generator:
else: else:
return self.func("TRIM", expression.this, expression.expression) return self.func("TRIM", expression.this, expression.expression)
def safeconcat_sql(self, expression: exp.SafeConcat) -> str: def convert_concat_args(self, expression: exp.Concat | exp.ConcatWs) -> t.List[exp.Expression]:
expressions = expression.expressions args = expression.expressions
if self.STRICT_STRING_CONCAT: if isinstance(expression, exp.ConcatWs):
expressions = (exp.cast(e, "text") for e in expressions) args = args[1:] # Skip the delimiter
if self.dialect.STRICT_STRING_CONCAT and expression.args.get("safe"):
args = [exp.cast(e, "text") for e in args]
if not self.dialect.CONCAT_COALESCE and expression.args.get("coalesce"):
args = [exp.func("coalesce", e, exp.Literal.string("")) for e in args]
return args
def concat_sql(self, expression: exp.Concat) -> str:
expressions = self.convert_concat_args(expression)
# Some dialects don't allow a single-argument CONCAT call
if not self.SUPPORTS_SINGLE_ARG_CONCAT and len(expressions) == 1:
return self.sql(expressions[0])
return self.func("CONCAT", *expressions) return self.func("CONCAT", *expressions)
def concatws_sql(self, expression: exp.ConcatWs) -> str:
return self.func(
"CONCAT_WS", seq_get(expression.expressions, 0), *self.convert_concat_args(expression)
)
def check_sql(self, expression: exp.Check) -> str: def check_sql(self, expression: exp.Check) -> str:
this = self.sql(expression, key="this") this = self.sql(expression, key="this")
return f"CHECK ({this})" return f"CHECK ({this})"
@ -2493,14 +2538,7 @@ class Generator:
actions = expression.args["actions"] actions = expression.args["actions"]
if isinstance(actions[0], exp.ColumnDef): if isinstance(actions[0], exp.ColumnDef):
if self.ALTER_TABLE_ADD_COLUMN_KEYWORD: actions = self.add_column_sql(expression)
actions = self.expressions(
expression,
key="actions",
prefix="ADD COLUMN ",
)
else:
actions = f"ADD {self.expressions(expression, key='actions')}"
elif isinstance(actions[0], exp.Schema): elif isinstance(actions[0], exp.Schema):
actions = self.expressions(expression, key="actions", prefix="ADD COLUMNS ") actions = self.expressions(expression, key="actions", prefix="ADD COLUMNS ")
elif isinstance(actions[0], exp.Delete): elif isinstance(actions[0], exp.Delete):
@ -2512,6 +2550,15 @@ class Generator:
only = " ONLY" if expression.args.get("only") else "" only = " ONLY" if expression.args.get("only") else ""
return f"ALTER TABLE{exists}{only} {self.sql(expression, 'this')} {actions}" return f"ALTER TABLE{exists}{only} {self.sql(expression, 'this')} {actions}"
def add_column_sql(self, expression: exp.AlterTable) -> str:
if self.ALTER_TABLE_INCLUDE_COLUMN_KEYWORD:
return self.expressions(
expression,
key="actions",
prefix="ADD COLUMN ",
)
return f"ADD {self.expressions(expression, key='actions', flat=True)}"
def droppartition_sql(self, expression: exp.DropPartition) -> str: def droppartition_sql(self, expression: exp.DropPartition) -> str:
expressions = self.expressions(expression) expressions = self.expressions(expression)
exists = " IF EXISTS " if expression.args.get("exists") else " " exists = " IF EXISTS " if expression.args.get("exists") else " "
@ -2551,14 +2598,31 @@ class Generator:
) )
def dpipe_sql(self, expression: exp.DPipe) -> str: def dpipe_sql(self, expression: exp.DPipe) -> str:
if self.dialect.STRICT_STRING_CONCAT and expression.args.get("safe"):
return self.func("CONCAT", *(exp.cast(e, "text") for e in expression.flatten()))
return self.binary(expression, "||") return self.binary(expression, "||")
def safedpipe_sql(self, expression: exp.SafeDPipe) -> str:
if self.STRICT_STRING_CONCAT:
return self.func("CONCAT", *(exp.cast(e, "text") for e in expression.flatten()))
return self.dpipe_sql(expression)
def div_sql(self, expression: exp.Div) -> str: def div_sql(self, expression: exp.Div) -> str:
l, r = expression.left, expression.right
if not self.dialect.SAFE_DIVISION and expression.args.get("safe"):
r.replace(exp.Nullif(this=r.copy(), expression=exp.Literal.number(0)))
if self.dialect.TYPED_DIVISION and not expression.args.get("typed"):
if not l.is_type(*exp.DataType.FLOAT_TYPES) and not r.is_type(
*exp.DataType.FLOAT_TYPES
):
l.replace(exp.cast(l.copy(), to=exp.DataType.Type.DOUBLE))
elif not self.dialect.TYPED_DIVISION and expression.args.get("typed"):
if l.is_type(*exp.DataType.INTEGER_TYPES) and r.is_type(*exp.DataType.INTEGER_TYPES):
return self.sql(
exp.cast(
l / r,
to=exp.DataType.Type.BIGINT,
)
)
return self.binary(expression, "/") return self.binary(expression, "/")
def overlaps_sql(self, expression: exp.Overlaps) -> str: def overlaps_sql(self, expression: exp.Overlaps) -> str:
@ -2573,6 +2637,9 @@ class Generator:
def eq_sql(self, expression: exp.EQ) -> str: def eq_sql(self, expression: exp.EQ) -> str:
return self.binary(expression, "=") return self.binary(expression, "=")
def propertyeq_sql(self, expression: exp.PropertyEQ) -> str:
return self.binary(expression, ":=")
def escape_sql(self, expression: exp.Escape) -> str: def escape_sql(self, expression: exp.Escape) -> str:
return self.binary(expression, "ESCAPE") return self.binary(expression, "ESCAPE")
@ -2641,10 +2708,13 @@ class Generator:
return self.cast_sql(expression, safe_prefix="TRY_") return self.cast_sql(expression, safe_prefix="TRY_")
def log_sql(self, expression: exp.Log) -> str: def log_sql(self, expression: exp.Log) -> str:
args = list(expression.args.values()) this = expression.this
if not self.LOG_BASE_FIRST: expr = expression.expression
args.reverse()
return self.func("LOG", *args) if not self.dialect.LOG_BASE_FIRST:
this, expr = expr, this
return self.func("LOG", this, expr)
def use_sql(self, expression: exp.Use) -> str: def use_sql(self, expression: exp.Use) -> str:
kind = self.sql(expression, "kind") kind = self.sql(expression, "kind")
@ -2696,7 +2766,9 @@ class Generator:
def format_time(self, expression: exp.Expression) -> t.Optional[str]: def format_time(self, expression: exp.Expression) -> t.Optional[str]:
return format_time( return format_time(
self.sql(expression, "format"), self.INVERSE_TIME_MAPPING, self.INVERSE_TIME_TRIE self.sql(expression, "format"),
self.dialect.INVERSE_TIME_MAPPING,
self.dialect.INVERSE_TIME_TRIE,
) )
def expressions( def expressions(
@ -2963,6 +3035,19 @@ class Generator:
parameters = self.sql(expression, "params_struct") parameters = self.sql(expression, "params_struct")
return self.func("PREDICT", model, table, parameters or None) return self.func("PREDICT", model, table, parameters or None)
def forin_sql(self, expression: exp.ForIn) -> str:
this = self.sql(expression, "this")
expression_sql = self.sql(expression, "expression")
return f"FOR {this} DO {expression_sql}"
def refresh_sql(self, expression: exp.Refresh) -> str:
this = self.sql(expression, "this")
table = "" if isinstance(expression.this, exp.Literal) else "TABLE "
return f"REFRESH {table}{this}"
def operator_sql(self, expression: exp.Operator) -> str:
return self.binary(expression, f"OPERATOR({self.sql(expression, 'operator')})")
def _simplify_unless_literal(self, expression: E) -> E: def _simplify_unless_literal(self, expression: E) -> E:
if not isinstance(expression, exp.Literal): if not isinstance(expression, exp.Literal):
from sqlglot.optimizer.simplify import simplify from sqlglot.optimizer.simplify import simplify
@ -2970,3 +3055,10 @@ class Generator:
expression = simplify(expression) expression = simplify(expression)
return expression return expression
def _ensure_string_if_null(self, values: t.List[exp.Expression]) -> t.List[exp.Expression]:
return [
exp.func("COALESCE", exp.cast(value, "text"), exp.Literal.string(""))
for value in values
if value
]

27
sqlglot/helper.py
View file

@ -1,5 +1,6 @@
from __future__ import annotations from __future__ import annotations
import datetime
import inspect import inspect
import logging import logging
import re import re
@ -283,7 +284,7 @@ def csv_reader(read_csv: exp.ReadCSV) -> t.Any:
file = open_file(read_csv.name) file = open_file(read_csv.name)
delimiter = "," delimiter = ","
args = iter(arg.name for arg in args) args = iter(arg.name for arg in args) # type: ignore
for k, v in zip(args, args): for k, v in zip(args, args):
if k == "delimiter": if k == "delimiter":
delimiter = v delimiter = v
@ -463,3 +464,27 @@ def merge_ranges(ranges: t.List[t.Tuple[A, A]]) -> t.List[t.Tuple[A, A]]:
merged.append((start, end)) merged.append((start, end))
return merged return merged
def is_iso_date(text: str) -> bool:
try:
datetime.date.fromisoformat(text)
return True
except ValueError:
return False
def is_iso_datetime(text: str) -> bool:
try:
datetime.datetime.fromisoformat(text)
return True
except ValueError:
return False
# Interval units that operate on date components
DATE_UNITS = {"day", "week", "month", "quarter", "year", "year_month"}
def is_date_unit(expression: t.Optional[exp.Expression]) -> bool:
return expression is not None and expression.name.lower() in DATE_UNITS

105
sqlglot/lineage.py
View file

@ -6,7 +6,7 @@ from dataclasses import dataclass, field
from sqlglot import Schema, exp, maybe_parse from sqlglot import Schema, exp, maybe_parse
from sqlglot.errors import SqlglotError from sqlglot.errors import SqlglotError
from sqlglot.optimizer import Scope, build_scope, qualify from sqlglot.optimizer import Scope, build_scope, find_all_in_scope, qualify
if t.TYPE_CHECKING: if t.TYPE_CHECKING:
from sqlglot.dialects.dialect import DialectType from sqlglot.dialects.dialect import DialectType
@ -29,8 +29,38 @@ class Node:
else: else:
yield d yield d
def to_html(self, **opts) -> LineageHTML: def to_html(self, dialect: DialectType = None, **opts) -> GraphHTML:
return LineageHTML(self, **opts) nodes = {}
edges = []
for node in self.walk():
if isinstance(node.expression, exp.Table):
label = f"FROM {node.expression.this}"
title = f"<pre>SELECT {node.name} FROM {node.expression.this}</pre>"
group = 1
else:
label = node.expression.sql(pretty=True, dialect=dialect)
source = node.source.transform(
lambda n: exp.Tag(this=n, prefix="<b>", postfix="</b>")
if n is node.expression
else n,
copy=False,
).sql(pretty=True, dialect=dialect)
title = f"<pre>{source}</pre>"
group = 0
node_id = id(node)
nodes[node_id] = {
"id": node_id,
"label": label,
"title": title,
"group": group,
}
for d in node.downstream:
edges.append({"from": node_id, "to": id(d)})
return GraphHTML(nodes, edges, **opts)
def lineage( def lineage(
@ -64,6 +94,7 @@ def lineage(
k: t.cast(exp.Subqueryable, maybe_parse(v, dialect=dialect)) k: t.cast(exp.Subqueryable, maybe_parse(v, dialect=dialect))
for k, v in sources.items() for k, v in sources.items()
}, },
dialect=dialect,
) )
qualified = qualify.qualify( qualified = qualify.qualify(
@ -129,17 +160,6 @@ def lineage(
return upstream return upstream
subquery = select.unalias()
if isinstance(subquery, exp.Subquery):
upstream = upstream or Node(name="SUBQUERY", source=scope.expression, expression=select)
scope = t.cast(Scope, build_scope(subquery.unnest()))
for select in subquery.named_selects:
to_node(select, scope=scope, upstream=upstream)
return upstream
if isinstance(scope.expression, exp.Select): if isinstance(scope.expression, exp.Select):
# For better ergonomics in our node labels, replace the full select with # For better ergonomics in our node labels, replace the full select with
# a version that has only the column we care about. # a version that has only the column we care about.
@ -156,16 +176,28 @@ def lineage(
expression=select, expression=select,
alias=alias or "", alias=alias or "",
) )
if upstream: if upstream:
upstream.downstream.append(node) upstream.downstream.append(node)
subquery_scopes = {
id(subquery_scope.expression): subquery_scope
for subquery_scope in scope.subquery_scopes
}
for subquery in find_all_in_scope(select, exp.Subqueryable):
subquery_scope = subquery_scopes[id(subquery)]
for name in subquery.named_selects:
to_node(name, scope=subquery_scope, upstream=node)
# if the select is a star add all scope sources as downstreams # if the select is a star add all scope sources as downstreams
if select.is_star: if select.is_star:
for source in scope.sources.values(): for source in scope.sources.values():
node.downstream.append(Node(name=select.sql(), source=source, expression=source)) node.downstream.append(Node(name=select.sql(), source=source, expression=source))
# Find all columns that went into creating this one to list their lineage nodes. # Find all columns that went into creating this one to list their lineage nodes.
source_columns = set(select.find_all(exp.Column)) source_columns = set(find_all_in_scope(select, exp.Column))
# If the source is a UDTF find columns used in the UTDF to generate the table # If the source is a UDTF find columns used in the UTDF to generate the table
if isinstance(source, exp.UDTF): if isinstance(source, exp.UDTF):
@ -192,20 +224,15 @@ def lineage(
return to_node(column if isinstance(column, str) else column.name, scope) return to_node(column if isinstance(column, str) else column.name, scope)
class LineageHTML: class GraphHTML:
"""Node to HTML generator using vis.js. """Node to HTML generator using vis.js.
https://visjs.github.io/vis-network/docs/network/ https://visjs.github.io/vis-network/docs/network/
""" """
def __init__( def __init__(
self, self, nodes: t.Dict, edges: t.List, imports: bool = True, options: t.Optional[t.Dict] = None
node: Node,
dialect: DialectType = None,
imports: bool = True,
**opts: t.Any,
): ):
self.node = node
self.imports = imports self.imports = imports
self.options = { self.options = {
@ -235,39 +262,11 @@ class LineageHTML:
"maximum": 300, "maximum": 300,
}, },
}, },
**opts, **(options or {}),
} }
self.nodes = {} self.nodes = nodes
self.edges = [] self.edges = edges
for node in node.walk():
if isinstance(node.expression, exp.Table):
label = f"FROM {node.expression.this}"
title = f"<pre>SELECT {node.name} FROM {node.expression.this}</pre>"
group = 1
else:
label = node.expression.sql(pretty=True, dialect=dialect)
source = node.source.transform(
lambda n: exp.Tag(this=n, prefix="<b>", postfix="</b>")
if n is node.expression
else n,
copy=False,
).sql(pretty=True, dialect=dialect)
title = f"<pre>{source}</pre>"
group = 0
node_id = id(node)
self.nodes[node_id] = {
"id": node_id,
"label": label,
"title": title,
"group": group,
}
for d in node.downstream:
self.edges.append({"from": node_id, "to": id(d)})
def __str__(self): def __str__(self):
nodes = json.dumps(list(self.nodes.values())) nodes = json.dumps(list(self.nodes.values()))

110
sqlglot/optimizer/annotate_types.py
View file

@ -1,12 +1,18 @@
from __future__ import annotations from __future__ import annotations
import datetime
import functools import functools
import typing as t import typing as t
from sqlglot import exp from sqlglot import exp
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.helper import ensure_list, seq_get, subclasses from sqlglot.helper import (
ensure_list,
is_date_unit,
is_iso_date,
is_iso_datetime,
seq_get,
subclasses,
)
from sqlglot.optimizer.scope import Scope, traverse_scope from sqlglot.optimizer.scope import Scope, traverse_scope
from sqlglot.schema import Schema, ensure_schema from sqlglot.schema import Schema, ensure_schema
@ -20,10 +26,6 @@ if t.TYPE_CHECKING:
] ]
# Interval units that operate on date components
DATE_UNITS = {"day", "week", "month", "quarter", "year", "year_month"}
def annotate_types( def annotate_types(
expression: E, expression: E,
schema: t.Optional[t.Dict | Schema] = None, schema: t.Optional[t.Dict | Schema] = None,
@ -60,43 +62,22 @@ def _annotate_with_type_lambda(data_type: exp.DataType.Type) -> t.Callable[[Type
return lambda self, e: self._annotate_with_type(e, data_type) return lambda self, e: self._annotate_with_type(e, data_type)
def _is_iso_date(text: str) -> bool: def _coerce_date_literal(l: exp.Expression, unit: t.Optional[exp.Expression]) -> exp.DataType.Type:
try:
datetime.date.fromisoformat(text)
return True
except ValueError:
return False
def _is_iso_datetime(text: str) -> bool:
try:
datetime.datetime.fromisoformat(text)
return True
except ValueError:
return False
def _coerce_literal_and_interval(l: exp.Expression, r: exp.Expression) -> exp.DataType.Type:
date_text = l.name date_text = l.name
unit = r.text("unit").lower() is_iso_date_ = is_iso_date(date_text)
is_iso_date = _is_iso_date(date_text) if is_iso_date_ and is_date_unit(unit):
if is_iso_date and unit in DATE_UNITS:
l.replace(exp.cast(l.copy(), to=exp.DataType.Type.DATE))
return exp.DataType.Type.DATE return exp.DataType.Type.DATE
# An ISO date is also an ISO datetime, but not vice versa # An ISO date is also an ISO datetime, but not vice versa
if is_iso_date or _is_iso_datetime(date_text): if is_iso_date_ or is_iso_datetime(date_text):
l.replace(exp.cast(l.copy(), to=exp.DataType.Type.DATETIME))
return exp.DataType.Type.DATETIME return exp.DataType.Type.DATETIME
return exp.DataType.Type.UNKNOWN return exp.DataType.Type.UNKNOWN
def _coerce_date_and_interval(l: exp.Expression, r: exp.Expression) -> exp.DataType.Type: def _coerce_date(l: exp.Expression, unit: t.Optional[exp.Expression]) -> exp.DataType.Type:
unit = r.text("unit").lower() if not is_date_unit(unit):
if unit not in DATE_UNITS:
return exp.DataType.Type.DATETIME return exp.DataType.Type.DATETIME
return l.type.this if l.type else exp.DataType.Type.UNKNOWN return l.type.this if l.type else exp.DataType.Type.UNKNOWN
@ -171,7 +152,6 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.Date, exp.Date,
exp.DateFromParts, exp.DateFromParts,
exp.DateStrToDate, exp.DateStrToDate,
exp.DateTrunc,
exp.DiToDate, exp.DiToDate,
exp.StrToDate, exp.StrToDate,
exp.TimeStrToDate, exp.TimeStrToDate,
@ -185,6 +165,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.DataType.Type.DOUBLE: { exp.DataType.Type.DOUBLE: {
exp.ApproxQuantile, exp.ApproxQuantile,
exp.Avg, exp.Avg,
exp.Div,
exp.Exp, exp.Exp,
exp.Ln, exp.Ln,
exp.Log, exp.Log,
@ -203,8 +184,8 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
}, },
exp.DataType.Type.INT: { exp.DataType.Type.INT: {
exp.Ceil, exp.Ceil,
exp.DateDiff,
exp.DatetimeDiff, exp.DatetimeDiff,
exp.DateDiff,
exp.Extract, exp.Extract,
exp.TimestampDiff, exp.TimestampDiff,
exp.TimeDiff, exp.TimeDiff,
@ -240,8 +221,6 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.GroupConcat, exp.GroupConcat,
exp.Initcap, exp.Initcap,
exp.Lower, exp.Lower,
exp.SafeConcat,
exp.SafeDPipe,
exp.Substring, exp.Substring,
exp.TimeToStr, exp.TimeToStr,
exp.TimeToTimeStr, exp.TimeToTimeStr,
@ -267,6 +246,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
for data_type, expressions in TYPE_TO_EXPRESSIONS.items() for data_type, expressions in TYPE_TO_EXPRESSIONS.items()
for expr_type in expressions for expr_type in expressions
}, },
exp.Abs: lambda self, e: self._annotate_by_args(e, "this"),
exp.Anonymous: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN), exp.Anonymous: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN),
exp.Array: lambda self, e: self._annotate_by_args(e, "expressions", array=True), exp.Array: lambda self, e: self._annotate_by_args(e, "expressions", array=True),
exp.ArrayAgg: lambda self, e: self._annotate_by_args(e, "this", array=True), exp.ArrayAgg: lambda self, e: self._annotate_by_args(e, "this", array=True),
@ -276,9 +256,11 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.Case: lambda self, e: self._annotate_by_args(e, "default", "ifs"), exp.Case: lambda self, e: self._annotate_by_args(e, "default", "ifs"),
exp.Coalesce: lambda self, e: self._annotate_by_args(e, "this", "expressions"), exp.Coalesce: lambda self, e: self._annotate_by_args(e, "this", "expressions"),
exp.DataType: lambda self, e: self._annotate_with_type(e, e.copy()), exp.DataType: lambda self, e: self._annotate_with_type(e, e.copy()),
exp.DateAdd: lambda self, e: self._annotate_dateadd(e), exp.DateAdd: lambda self, e: self._annotate_timeunit(e),
exp.DateSub: lambda self, e: self._annotate_dateadd(e), exp.DateSub: lambda self, e: self._annotate_timeunit(e),
exp.DateTrunc: lambda self, e: self._annotate_timeunit(e),
exp.Distinct: lambda self, e: self._annotate_by_args(e, "expressions"), exp.Distinct: lambda self, e: self._annotate_by_args(e, "expressions"),
exp.Div: lambda self, e: self._annotate_div(e),
exp.Filter: lambda self, e: self._annotate_by_args(e, "this"), exp.Filter: lambda self, e: self._annotate_by_args(e, "this"),
exp.If: lambda self, e: self._annotate_by_args(e, "true", "false"), exp.If: lambda self, e: self._annotate_by_args(e, "true", "false"),
exp.Interval: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.INTERVAL), exp.Interval: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.INTERVAL),
@ -288,6 +270,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.Max: lambda self, e: self._annotate_by_args(e, "this", "expressions"), exp.Max: lambda self, e: self._annotate_by_args(e, "this", "expressions"),
exp.Min: lambda self, e: self._annotate_by_args(e, "this", "expressions"), exp.Min: lambda self, e: self._annotate_by_args(e, "this", "expressions"),
exp.Null: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.NULL), exp.Null: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.NULL),
exp.Nullif: lambda self, e: self._annotate_by_args(e, "this", "expression"),
exp.Slice: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN), exp.Slice: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN),
exp.Sum: lambda self, e: self._annotate_by_args(e, "this", "expressions", promote=True), exp.Sum: lambda self, e: self._annotate_by_args(e, "this", "expressions", promote=True),
exp.TryCast: lambda self, e: self._annotate_with_type(e, e.args["to"]), exp.TryCast: lambda self, e: self._annotate_with_type(e, e.args["to"]),
@ -306,13 +289,27 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
BINARY_COERCIONS: BinaryCoercions = { BINARY_COERCIONS: BinaryCoercions = {
**swap_all( **swap_all(
{ {
(t, exp.DataType.Type.INTERVAL): _coerce_literal_and_interval (t, exp.DataType.Type.INTERVAL): lambda l, r: _coerce_date_literal(
l, r.args.get("unit")
)
for t in exp.DataType.TEXT_TYPES for t in exp.DataType.TEXT_TYPES
} }
), ),
**swap_all( **swap_all(
{ {
(exp.DataType.Type.DATE, exp.DataType.Type.INTERVAL): _coerce_date_and_interval, # text + numeric will yield the numeric type to match most dialects' semantics
(text, numeric): lambda l, r: t.cast(
exp.DataType.Type, l.type if l.type in exp.DataType.NUMERIC_TYPES else r.type
)
for text in exp.DataType.TEXT_TYPES
for numeric in exp.DataType.NUMERIC_TYPES
}
),
**swap_all(
{
(exp.DataType.Type.DATE, exp.DataType.Type.INTERVAL): lambda l, r: _coerce_date(
l, r.args.get("unit")
),
} }
), ),
} }
@ -511,18 +508,17 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
return expression return expression
def _annotate_dateadd(self, expression: exp.IntervalOp) -> exp.IntervalOp: def _annotate_timeunit(
self, expression: exp.TimeUnit | exp.DateTrunc
) -> exp.TimeUnit | exp.DateTrunc:
self._annotate_args(expression) self._annotate_args(expression)
if expression.this.type.this in exp.DataType.TEXT_TYPES: if expression.this.type.this in exp.DataType.TEXT_TYPES:
datatype = _coerce_literal_and_interval(expression.this, expression.interval()) datatype = _coerce_date_literal(expression.this, expression.unit)
elif ( elif expression.this.type.this in exp.DataType.TEMPORAL_TYPES:
expression.this.type.is_type(exp.DataType.Type.DATE) datatype = _coerce_date(expression.this, expression.unit)
and expression.text("unit").lower() not in DATE_UNITS
):
datatype = exp.DataType.Type.DATETIME
else: else:
datatype = expression.this.type datatype = exp.DataType.Type.UNKNOWN
self._set_type(expression, datatype) self._set_type(expression, datatype)
return expression return expression
@ -547,3 +543,19 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
self._set_type(expression, exp.DataType.Type.UNKNOWN) self._set_type(expression, exp.DataType.Type.UNKNOWN)
return expression return expression
def _annotate_div(self, expression: exp.Div) -> exp.Div:
self._annotate_args(expression)
left_type, right_type = expression.left.type.this, expression.right.type.this # type: ignore
if (
expression.args.get("typed")
and left_type in exp.DataType.INTEGER_TYPES
and right_type in exp.DataType.INTEGER_TYPES
):
self._set_type(expression, exp.DataType.Type.BIGINT)
else:
self._set_type(expression, self._maybe_coerce(left_type, right_type))
return expression

85
sqlglot/optimizer/canonicalize.py
View file

@ -1,8 +1,10 @@
from __future__ import annotations from __future__ import annotations
import itertools import itertools
import typing as t
from sqlglot import exp from sqlglot import exp
from sqlglot.helper import is_date_unit, is_iso_date, is_iso_datetime
def canonicalize(expression: exp.Expression) -> exp.Expression: def canonicalize(expression: exp.Expression) -> exp.Expression:
@ -20,7 +22,7 @@ def canonicalize(expression: exp.Expression) -> exp.Expression:
expression = replace_date_funcs(expression) expression = replace_date_funcs(expression)
expression = coerce_type(expression) expression = coerce_type(expression)
expression = remove_redundant_casts(expression) expression = remove_redundant_casts(expression)
expression = ensure_bool_predicates(expression) expression = ensure_bools(expression, _replace_int_predicate)
expression = remove_ascending_order(expression) expression = remove_ascending_order(expression)
return expression return expression
@ -40,8 +42,22 @@ def replace_date_funcs(node: exp.Expression) -> exp.Expression:
return node return node
COERCIBLE_DATE_OPS = (
exp.Add,
exp.Sub,
exp.EQ,
exp.NEQ,
exp.GT,
exp.GTE,
exp.LT,
exp.LTE,
exp.NullSafeEQ,
exp.NullSafeNEQ,
)
def coerce_type(node: exp.Expression) -> exp.Expression: def coerce_type(node: exp.Expression) -> exp.Expression:
if isinstance(node, exp.Binary): if isinstance(node, COERCIBLE_DATE_OPS):
_coerce_date(node.left, node.right) _coerce_date(node.left, node.right)
elif isinstance(node, exp.Between): elif isinstance(node, exp.Between):
_coerce_date(node.this, node.args["low"]) _coerce_date(node.this, node.args["low"])
@ -49,6 +65,10 @@ def coerce_type(node: exp.Expression) -> exp.Expression:
*exp.DataType.TEMPORAL_TYPES *exp.DataType.TEMPORAL_TYPES
): ):
_replace_cast(node.expression, exp.DataType.Type.DATETIME) _replace_cast(node.expression, exp.DataType.Type.DATETIME)
elif isinstance(node, (exp.DateAdd, exp.DateSub, exp.DateTrunc)):
_coerce_timeunit_arg(node.this, node.unit)
elif isinstance(node, exp.DateDiff):
_coerce_datediff_args(node)
return node return node
@ -64,17 +84,21 @@ def remove_redundant_casts(expression: exp.Expression) -> exp.Expression:
return expression return expression
def ensure_bool_predicates(expression: exp.Expression) -> exp.Expression: def ensure_bools(
expression: exp.Expression, replace_func: t.Callable[[exp.Expression], None]
) -> exp.Expression:
if isinstance(expression, exp.Connector): if isinstance(expression, exp.Connector):
_replace_int_predicate(expression.left) replace_func(expression.left)
_replace_int_predicate(expression.right) replace_func(expression.right)
elif isinstance(expression, exp.Not):
elif isinstance(expression, (exp.Where, exp.Having)) or ( replace_func(expression.this)
# We can't replace num in CASE x WHEN num ..., because it's not the full predicate # We can't replace num in CASE x WHEN num ..., because it's not the full predicate
isinstance(expression, exp.If) elif isinstance(expression, exp.If) and not (
and not (isinstance(expression.parent, exp.Case) and expression.parent.this) isinstance(expression.parent, exp.Case) and expression.parent.this
): ):
_replace_int_predicate(expression.this) replace_func(expression.this)
elif isinstance(expression, (exp.Where, exp.Having)):
replace_func(expression.this)
return expression return expression
@ -89,22 +113,59 @@ def remove_ascending_order(expression: exp.Expression) -> exp.Expression:
def _coerce_date(a: exp.Expression, b: exp.Expression) -> None: def _coerce_date(a: exp.Expression, b: exp.Expression) -> None:
for a, b in itertools.permutations([a, b]): for a, b in itertools.permutations([a, b]):
if isinstance(b, exp.Interval):
a = _coerce_timeunit_arg(a, b.unit)
if ( if (
a.type a.type
and a.type.this == exp.DataType.Type.DATE and a.type.this == exp.DataType.Type.DATE
and b.type and b.type
and b.type.this not in (exp.DataType.Type.DATE, exp.DataType.Type.INTERVAL) and b.type.this
not in (
exp.DataType.Type.DATE,
exp.DataType.Type.INTERVAL,
)
): ):
_replace_cast(b, exp.DataType.Type.DATE) _replace_cast(b, exp.DataType.Type.DATE)
def _coerce_timeunit_arg(arg: exp.Expression, unit: t.Optional[exp.Expression]) -> exp.Expression:
if not arg.type:
return arg
if arg.type.this in exp.DataType.TEXT_TYPES:
date_text = arg.name
is_iso_date_ = is_iso_date(date_text)
if is_iso_date_ and is_date_unit(unit):
return arg.replace(exp.cast(arg.copy(), to=exp.DataType.Type.DATE))
# An ISO date is also an ISO datetime, but not vice versa
if is_iso_date_ or is_iso_datetime(date_text):
return arg.replace(exp.cast(arg.copy(), to=exp.DataType.Type.DATETIME))
elif arg.type.this == exp.DataType.Type.DATE and not is_date_unit(unit):
return arg.replace(exp.cast(arg.copy(), to=exp.DataType.Type.DATETIME))
return arg
def _coerce_datediff_args(node: exp.DateDiff) -> None:
for e in (node.this, node.expression):
if e.type.this not in exp.DataType.TEMPORAL_TYPES:
e.replace(exp.cast(e.copy(), to=exp.DataType.Type.DATETIME))
def _replace_cast(node: exp.Expression, to: exp.DataType.Type) -> None: def _replace_cast(node: exp.Expression, to: exp.DataType.Type) -> None:
node.replace(exp.cast(node.copy(), to=to)) node.replace(exp.cast(node.copy(), to=to))
# this was originally designed for presto, there is a similar transform for tsql
# this is different in that it only operates on int types, this is because
# presto has a boolean type whereas tsql doesn't (people use bits)
# with y as (select true as x) select x = 0 FROM y -- illegal presto query
def _replace_int_predicate(expression: exp.Expression) -> None: def _replace_int_predicate(expression: exp.Expression) -> None:
if isinstance(expression, exp.Coalesce): if isinstance(expression, exp.Coalesce):
for _, child in expression.iter_expressions(): for _, child in expression.iter_expressions():
_replace_int_predicate(child) _replace_int_predicate(child)
elif expression.type and expression.type.this in exp.DataType.INTEGER_TYPES: elif expression.type and expression.type.this in exp.DataType.INTEGER_TYPES:
expression.replace(exp.NEQ(this=expression.copy(), expression=exp.Literal.number(0))) expression.replace(expression.neq(0))

4
sqlglot/optimizer/merge_subqueries.py
View file

@ -186,13 +186,13 @@ def _mergeable(outer_scope, inner_scope, leave_tables_isolated, from_or_join):
and not ( and not (
isinstance(from_or_join, exp.Join) isinstance(from_or_join, exp.Join)
and inner_select.args.get("where") and inner_select.args.get("where")
and from_or_join.side in {"FULL", "LEFT", "RIGHT"} and from_or_join.side in ("FULL", "LEFT", "RIGHT")
) )
and not ( and not (
isinstance(from_or_join, exp.From) isinstance(from_or_join, exp.From)
and inner_select.args.get("where") and inner_select.args.get("where")
and any( and any(
j.side in {"FULL", "RIGHT"} for j in outer_scope.expression.args.get("joins", []) j.side in ("FULL", "RIGHT") for j in outer_scope.expression.args.get("joins", [])
) )
) )
and not _outer_select_joins_on_inner_select_join() and not _outer_select_joins_on_inner_select_join()

6
sqlglot/optimizer/normalize_identifiers.py
View file

@ -13,7 +13,7 @@ def normalize_identifiers(expression: E, dialect: DialectType = None) -> E:
@t.overload @t.overload
def normalize_identifiers(expression: str, dialect: DialectType = None) -> exp.Expression: def normalize_identifiers(expression: str, dialect: DialectType = None) -> exp.Identifier:
... ...
@ -48,11 +48,11 @@ def normalize_identifiers(expression, dialect=None):
Returns: Returns:
The transformed expression. The transformed expression.
""" """
dialect = Dialect.get_or_raise(dialect)
if isinstance(expression, str): if isinstance(expression, str):
expression = exp.parse_identifier(expression, dialect=dialect) expression = exp.parse_identifier(expression, dialect=dialect)
dialect = Dialect.get_or_raise(dialect)
def _normalize(node: E) -> E: def _normalize(node: E) -> E:
if not node.meta.get("case_sensitive"): if not node.meta.get("case_sensitive"):
exp.replace_children(node, _normalize) exp.replace_children(node, _normalize)

4
sqlglot/optimizer/optimizer.py
View file

@ -42,8 +42,8 @@ RULES = (
def optimize( def optimize(
expression: str | exp.Expression, expression: str | exp.Expression,
schema: t.Optional[dict | Schema] = None, schema: t.Optional[dict | Schema] = None,
db: t.Optional[str] = None, db: t.Optional[str | exp.Identifier] = None,
catalog: t.Optional[str] = None, catalog: t.Optional[str | exp.Identifier] = None,
dialect: DialectType = None, dialect: DialectType = None,
rules: t.Sequence[t.Callable] = RULES, rules: t.Sequence[t.Callable] = RULES,
**kwargs, **kwargs,

41
sqlglot/optimizer/qualify_columns.py
View file

@ -8,7 +8,7 @@ from sqlglot._typing import E
from sqlglot.dialects.dialect import Dialect, DialectType from sqlglot.dialects.dialect import Dialect, DialectType
from sqlglot.errors import OptimizeError from sqlglot.errors import OptimizeError
from sqlglot.helper import seq_get from sqlglot.helper import seq_get
from sqlglot.optimizer.scope import Scope, traverse_scope, walk_in_scope from sqlglot.optimizer.scope import Scope, build_scope, traverse_scope, walk_in_scope
from sqlglot.optimizer.simplify import simplify_parens from sqlglot.optimizer.simplify import simplify_parens
from sqlglot.schema import Schema, ensure_schema from sqlglot.schema import Schema, ensure_schema
@ -58,7 +58,7 @@ def qualify_columns(
if not isinstance(scope.expression, exp.UDTF): if not isinstance(scope.expression, exp.UDTF):
_expand_stars(scope, resolver, using_column_tables, pseudocolumns) _expand_stars(scope, resolver, using_column_tables, pseudocolumns)
_qualify_outputs(scope) qualify_outputs(scope)
_expand_group_by(scope) _expand_group_by(scope)
_expand_order_by(scope, resolver) _expand_order_by(scope, resolver)
@ -237,7 +237,7 @@ def _expand_order_by(scope: Scope, resolver: Resolver) -> None:
ordereds = order.expressions ordereds = order.expressions
for ordered, new_expression in zip( for ordered, new_expression in zip(
ordereds, ordereds,
_expand_positional_references(scope, (o.this for o in ordereds)), _expand_positional_references(scope, (o.this for o in ordereds), alias=True),
): ):
for agg in ordered.find_all(exp.AggFunc): for agg in ordered.find_all(exp.AggFunc):
for col in agg.find_all(exp.Column): for col in agg.find_all(exp.Column):
@ -259,17 +259,23 @@ def _expand_order_by(scope: Scope, resolver: Resolver) -> None:
) )
def _expand_positional_references(scope: Scope, expressions: t.Iterable[E]) -> t.List[E]: def _expand_positional_references(
new_nodes = [] scope: Scope, expressions: t.Iterable[exp.Expression], alias: bool = False
) -> t.List[exp.Expression]:
new_nodes: t.List[exp.Expression] = []
for node in expressions: for node in expressions:
if node.is_int: if node.is_int:
select = _select_by_pos(scope, t.cast(exp.Literal, node)).this select = _select_by_pos(scope, t.cast(exp.Literal, node))
if alias:
new_nodes.append(exp.column(select.args["alias"].copy()))
else:
select = select.this
if isinstance(select, exp.Literal): if isinstance(select, exp.Literal):
new_nodes.append(node) new_nodes.append(node)
else: else:
new_nodes.append(select.copy()) new_nodes.append(select.copy())
scope.clear_cache()
else: else:
new_nodes.append(node) new_nodes.append(node)
@ -307,7 +313,9 @@ def _qualify_columns(scope: Scope, resolver: Resolver) -> None:
if column_table: if column_table:
column.set("table", column_table) column.set("table", column_table)
elif column_table not in scope.sources and ( elif column_table not in scope.sources and (
not scope.parent or column_table not in scope.parent.sources not scope.parent
or column_table not in scope.parent.sources
or not scope.is_correlated_subquery
): ):
# structs are used like tables (e.g. "struct"."field"), so they need to be qualified # structs are used like tables (e.g. "struct"."field"), so they need to be qualified
# separately and represented as dot(dot(...(<table>.<column>, field1), field2, ...)) # separately and represented as dot(dot(...(<table>.<column>, field1), field2, ...))
@ -381,15 +389,18 @@ def _expand_stars(
columns = [name for name in columns if name.upper() not in pseudocolumns] columns = [name for name in columns if name.upper() not in pseudocolumns]
if columns and "*" not in columns: if columns and "*" not in columns:
table_id = id(table)
columns_to_exclude = except_columns.get(table_id) or set()
if pivot and has_pivoted_source and pivot_columns and pivot_output_columns: if pivot and has_pivoted_source and pivot_columns and pivot_output_columns:
implicit_columns = [col for col in columns if col not in pivot_columns] implicit_columns = [col for col in columns if col not in pivot_columns]
new_selections.extend( new_selections.extend(
exp.alias_(exp.column(name, table=pivot.alias), name, copy=False) exp.alias_(exp.column(name, table=pivot.alias), name, copy=False)
for name in implicit_columns + pivot_output_columns for name in implicit_columns + pivot_output_columns
if name not in columns_to_exclude
) )
continue continue
table_id = id(table)
for name in columns: for name in columns:
if name in using_column_tables and table in using_column_tables[name]: if name in using_column_tables and table in using_column_tables[name]:
if name in coalesced_columns: if name in coalesced_columns:
@ -406,7 +417,7 @@ def _expand_stars(
copy=False, copy=False,
) )
) )
elif name not in except_columns.get(table_id, set()): elif name not in columns_to_exclude:
alias_ = replace_columns.get(table_id, {}).get(name, name) alias_ = replace_columns.get(table_id, {}).get(name, name)
column = exp.column(name, table=table) column = exp.column(name, table=table)
new_selections.append( new_selections.append(
@ -448,10 +459,16 @@ def _add_replace_columns(
replace_columns[id(table)] = columns replace_columns[id(table)] = columns
def _qualify_outputs(scope: Scope) -> None: def qualify_outputs(scope_or_expression: Scope | exp.Expression) -> None:
"""Ensure all output columns are aliased""" """Ensure all output columns are aliased"""
new_selections = [] if isinstance(scope_or_expression, exp.Expression):
scope = build_scope(scope_or_expression)
if not isinstance(scope, Scope):
return
else:
scope = scope_or_expression
new_selections = []
for i, (selection, aliased_column) in enumerate( for i, (selection, aliased_column) in enumerate(
itertools.zip_longest(scope.expression.selects, scope.outer_column_list) itertools.zip_longest(scope.expression.selects, scope.outer_column_list)
): ):

15
sqlglot/optimizer/qualify_tables.py
View file

@ -1,8 +1,11 @@
from __future__ import annotations
import itertools import itertools
import typing as t import typing as t
from sqlglot import alias, exp from sqlglot import alias, exp
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.dialects.dialect import DialectType
from sqlglot.helper import csv_reader, name_sequence from sqlglot.helper import csv_reader, name_sequence
from sqlglot.optimizer.scope import Scope, traverse_scope from sqlglot.optimizer.scope import Scope, traverse_scope
from sqlglot.schema import Schema from sqlglot.schema import Schema
@ -10,9 +13,10 @@ from sqlglot.schema import Schema
def qualify_tables( def qualify_tables(
expression: E, expression: E,
db: t.Optional[str] = None, db: t.Optional[str | exp.Identifier] = None,
catalog: t.Optional[str] = None, catalog: t.Optional[str | exp.Identifier] = None,
schema: t.Optional[Schema] = None, schema: t.Optional[Schema] = None,
dialect: DialectType = None,
) -> E: ) -> E:
""" """
Rewrite sqlglot AST to have fully qualified tables. Join constructs such as Rewrite sqlglot AST to have fully qualified tables. Join constructs such as
@ -33,11 +37,14 @@ def qualify_tables(
db: Database name db: Database name
catalog: Catalog name catalog: Catalog name
schema: A schema to populate schema: A schema to populate
dialect: The dialect to parse catalog and schema into.
Returns: Returns:
The qualified expression. The qualified expression.
""" """
next_alias_name = name_sequence("_q_") next_alias_name = name_sequence("_q_")
db = exp.parse_identifier(db, dialect=dialect) if db else None
catalog = exp.parse_identifier(catalog, dialect=dialect) if catalog else None
for scope in traverse_scope(expression): for scope in traverse_scope(expression):
for derived_table in itertools.chain(scope.ctes, scope.derived_tables): for derived_table in itertools.chain(scope.ctes, scope.derived_tables):
@ -61,9 +68,9 @@ def qualify_tables(
if isinstance(source, exp.Table): if isinstance(source, exp.Table):
if isinstance(source.this, exp.Identifier): if isinstance(source.this, exp.Identifier):
if not source.args.get("db"): if not source.args.get("db"):
source.set("db", exp.to_identifier(db)) source.set("db", db)
if not source.args.get("catalog") and source.args.get("db"): if not source.args.get("catalog") and source.args.get("db"):
source.set("catalog", exp.to_identifier(catalog)) source.set("catalog", catalog)
if not source.alias: if not source.alias:
# Mutates the source by attaching an alias to it # Mutates the source by attaching an alias to it

2
sqlglot/optimizer/scope.py
View file

@ -1,3 +1,5 @@
from __future__ import annotations
import itertools import itertools
import logging import logging
import typing as t import typing as t

71
sqlglot/optimizer/simplify.py
View file

@ -507,6 +507,9 @@ def simplify_literals(expression, root=True):
return exp.Literal.number(value[1:]) return exp.Literal.number(value[1:])
return exp.Literal.number(f"-{value}") return exp.Literal.number(f"-{value}")
if type(expression) in INVERSE_DATE_OPS:
return _simplify_binary(expression, expression.this, expression.interval()) or expression
return expression return expression
@ -530,22 +533,24 @@ def _simplify_binary(expression, a, b):
return exp.null() return exp.null()
if a.is_number and b.is_number: if a.is_number and b.is_number:
a = int(a.name) if a.is_int else Decimal(a.name) num_a = int(a.name) if a.is_int else Decimal(a.name)
b = int(b.name) if b.is_int else Decimal(b.name) num_b = int(b.name) if b.is_int else Decimal(b.name)
if isinstance(expression, exp.Add): if isinstance(expression, exp.Add):
return exp.Literal.number(a + b) return exp.Literal.number(num_a + num_b)
if isinstance(expression, exp.Sub):
return exp.Literal.number(a - b)
if isinstance(expression, exp.Mul): if isinstance(expression, exp.Mul):
return exp.Literal.number(a * b) return exp.Literal.number(num_a * num_b)
# We only simplify Sub, Div if a and b have the same parent because they're not associative
if isinstance(expression, exp.Sub):
return exp.Literal.number(num_a - num_b) if a.parent is b.parent else None
if isinstance(expression, exp.Div): if isinstance(expression, exp.Div):
# engines have differing int div behavior so intdiv is not safe # engines have differing int div behavior so intdiv is not safe
if isinstance(a, int) and isinstance(b, int): if (isinstance(num_a, int) and isinstance(num_b, int)) or a.parent is not b.parent:
return None return None
return exp.Literal.number(a / b) return exp.Literal.number(num_a / num_b)
boolean = eval_boolean(expression, a, b) boolean = eval_boolean(expression, num_a, num_b)
if boolean: if boolean:
return boolean return boolean
@ -557,15 +562,21 @@ def _simplify_binary(expression, a, b):
elif _is_date_literal(a) and isinstance(b, exp.Interval): elif _is_date_literal(a) and isinstance(b, exp.Interval):
a, b = extract_date(a), extract_interval(b) a, b = extract_date(a), extract_interval(b)
if a and b: if a and b:
if isinstance(expression, exp.Add): if isinstance(expression, (exp.Add, exp.DateAdd, exp.DatetimeAdd)):
return date_literal(a + b) return date_literal(a + b)
if isinstance(expression, exp.Sub): if isinstance(expression, (exp.Sub, exp.DateSub, exp.DatetimeSub)):
return date_literal(a - b) return date_literal(a - b)
elif isinstance(a, exp.Interval) and _is_date_literal(b): elif isinstance(a, exp.Interval) and _is_date_literal(b):
a, b = extract_interval(a), extract_date(b) a, b = extract_interval(a), extract_date(b)
# you cannot subtract a date from an interval # you cannot subtract a date from an interval
if a and b and isinstance(expression, exp.Add): if a and b and isinstance(expression, exp.Add):
return date_literal(a + b) return date_literal(a + b)
elif _is_date_literal(a) and _is_date_literal(b):
if isinstance(expression, exp.Predicate):
a, b = extract_date(a), extract_date(b)
boolean = eval_boolean(expression, a, b)
if boolean:
return boolean
return None return None
@ -590,6 +601,11 @@ def simplify_parens(expression):
return expression return expression
NONNULL_CONSTANTS = (
exp.Literal,
exp.Boolean,
)
CONSTANTS = ( CONSTANTS = (
exp.Literal, exp.Literal,
exp.Boolean, exp.Boolean,
@ -597,11 +613,19 @@ CONSTANTS = (
) )
def _is_nonnull_constant(expression: exp.Expression) -> bool:
return isinstance(expression, NONNULL_CONSTANTS) or _is_date_literal(expression)
def _is_constant(expression: exp.Expression) -> bool:
return isinstance(expression, CONSTANTS) or _is_date_literal(expression)
def simplify_coalesce(expression): def simplify_coalesce(expression):
# COALESCE(x) -> x # COALESCE(x) -> x
if ( if (
isinstance(expression, exp.Coalesce) isinstance(expression, exp.Coalesce)
and not expression.expressions and (not expression.expressions or _is_nonnull_constant(expression.this))
# COALESCE is also used as a Spark partitioning hint # COALESCE is also used as a Spark partitioning hint
and not isinstance(expression.parent, exp.Hint) and not isinstance(expression.parent, exp.Hint)
): ):
@ -621,12 +645,12 @@ def simplify_coalesce(expression):
# This transformation is valid for non-constants, # This transformation is valid for non-constants,
# but it really only does anything if they are both constants. # but it really only does anything if they are both constants.
if not isinstance(other, CONSTANTS): if not _is_constant(other):
return expression return expression
# Find the first constant arg # Find the first constant arg
for arg_index, arg in enumerate(coalesce.expressions): for arg_index, arg in enumerate(coalesce.expressions):
if isinstance(arg, CONSTANTS): if _is_constant(other):
break break
else: else:
return expression return expression
@ -656,7 +680,6 @@ def simplify_coalesce(expression):
CONCATS = (exp.Concat, exp.DPipe) CONCATS = (exp.Concat, exp.DPipe)
SAFE_CONCATS = (exp.SafeConcat, exp.SafeDPipe)
def simplify_concat(expression): def simplify_concat(expression):
@ -672,10 +695,15 @@ def simplify_concat(expression):
sep_expr, *expressions = expression.expressions sep_expr, *expressions = expression.expressions
sep = sep_expr.name sep = sep_expr.name
concat_type = exp.ConcatWs concat_type = exp.ConcatWs
args = {}
else: else:
expressions = expression.expressions expressions = expression.expressions
sep = "" sep = ""
concat_type = exp.SafeConcat if isinstance(expression, SAFE_CONCATS) else exp.Concat concat_type = exp.Concat
args = {
"safe": expression.args.get("safe"),
"coalesce": expression.args.get("coalesce"),
}
new_args = [] new_args = []
for is_string_group, group in itertools.groupby( for is_string_group, group in itertools.groupby(
@ -692,7 +720,7 @@ def simplify_concat(expression):
if concat_type is exp.ConcatWs: if concat_type is exp.ConcatWs:
new_args = [sep_expr] + new_args new_args = [sep_expr] + new_args
return concat_type(expressions=new_args) return concat_type(expressions=new_args, **args)
def simplify_conditionals(expression): def simplify_conditionals(expression):
@ -947,7 +975,7 @@ def cast_value(value: t.Any, to: exp.DataType) -> t.Optional[t.Union[datetime.da
def extract_date(cast: exp.Expression) -> t.Optional[t.Union[datetime.date, datetime.date]]: def extract_date(cast: exp.Expression) -> t.Optional[t.Union[datetime.date, datetime.date]]:
if isinstance(cast, exp.Cast): if isinstance(cast, exp.Cast):
to = cast.to to = cast.to
elif isinstance(cast, exp.TsOrDsToDate): elif isinstance(cast, exp.TsOrDsToDate) and not cast.args.get("format"):
to = exp.DataType.build(exp.DataType.Type.DATE) to = exp.DataType.build(exp.DataType.Type.DATE)
else: else:
return None return None
@ -966,12 +994,11 @@ def _is_date_literal(expression: exp.Expression) -> bool:
def extract_interval(expression): def extract_interval(expression):
try:
n = int(expression.name) n = int(expression.name)
unit = expression.text("unit").lower() unit = expression.text("unit").lower()
try:
return interval(unit, n) return interval(unit, n)
except (UnsupportedUnit, ModuleNotFoundError): except (UnsupportedUnit, ModuleNotFoundError, ValueError):
return None return None
@ -1099,8 +1126,6 @@ GEN_MAP = {
exp.DataType: lambda e: f"{e.this.name} {gen(tuple(e.args.values())[1:])}", exp.DataType: lambda e: f"{e.this.name} {gen(tuple(e.args.values())[1:])}",
exp.Div: lambda e: _binary(e, "/"), exp.Div: lambda e: _binary(e, "/"),
exp.Dot: lambda e: _binary(e, "."), exp.Dot: lambda e: _binary(e, "."),
exp.DPipe: lambda e: _binary(e, "||"),
exp.SafeDPipe: lambda e: _binary(e, "||"),
exp.EQ: lambda e: _binary(e, "="), exp.EQ: lambda e: _binary(e, "="),
exp.GT: lambda e: _binary(e, ">"), exp.GT: lambda e: _binary(e, ">"),
exp.GTE: lambda e: _binary(e, ">="), exp.GTE: lambda e: _binary(e, ">="),

293
sqlglot/parser.py
View file

@ -13,6 +13,7 @@ from sqlglot.trie import TrieResult, in_trie, new_trie
if t.TYPE_CHECKING: if t.TYPE_CHECKING:
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.dialects.dialect import Dialect, DialectType
logger = logging.getLogger("sqlglot") logger = logging.getLogger("sqlglot")
@ -46,6 +47,19 @@ def binary_range_parser(
) )
def parse_logarithm(args: t.List, dialect: Dialect) -> exp.Func:
# Default argument order is base, expression
this = seq_get(args, 0)
expression = seq_get(args, 1)
if expression:
if not dialect.LOG_BASE_FIRST:
this, expression = expression, this
return exp.Log(this=this, expression=expression)
return (exp.Ln if dialect.parser_class.LOG_DEFAULTS_TO_LN else exp.Log)(this=this)
class _Parser(type): class _Parser(type):
def __new__(cls, clsname, bases, attrs): def __new__(cls, clsname, bases, attrs):
klass = super().__new__(cls, clsname, bases, attrs) klass = super().__new__(cls, clsname, bases, attrs)
@ -72,13 +86,24 @@ class Parser(metaclass=_Parser):
""" """
FUNCTIONS: t.Dict[str, t.Callable] = { FUNCTIONS: t.Dict[str, t.Callable] = {
**{name: f.from_arg_list for f in exp.ALL_FUNCTIONS for name in f.sql_names()}, **{name: func.from_arg_list for name, func in exp.FUNCTION_BY_NAME.items()},
"CONCAT": lambda args, dialect: exp.Concat(
expressions=args,
safe=not dialect.STRICT_STRING_CONCAT,
coalesce=dialect.CONCAT_COALESCE,
),
"CONCAT_WS": lambda args, dialect: exp.ConcatWs(
expressions=args,
safe=not dialect.STRICT_STRING_CONCAT,
coalesce=dialect.CONCAT_COALESCE,
),
"DATE_TO_DATE_STR": lambda args: exp.Cast( "DATE_TO_DATE_STR": lambda args: exp.Cast(
this=seq_get(args, 0), this=seq_get(args, 0),
to=exp.DataType(this=exp.DataType.Type.TEXT), to=exp.DataType(this=exp.DataType.Type.TEXT),
), ),
"GLOB": lambda args: exp.Glob(this=seq_get(args, 1), expression=seq_get(args, 0)), "GLOB": lambda args: exp.Glob(this=seq_get(args, 1), expression=seq_get(args, 0)),
"LIKE": parse_like, "LIKE": parse_like,
"LOG": parse_logarithm,
"TIME_TO_TIME_STR": lambda args: exp.Cast( "TIME_TO_TIME_STR": lambda args: exp.Cast(
this=seq_get(args, 0), this=seq_get(args, 0),
to=exp.DataType(this=exp.DataType.Type.TEXT), to=exp.DataType(this=exp.DataType.Type.TEXT),
@ -229,7 +254,7 @@ class Parser(metaclass=_Parser):
TokenType.SOME: exp.Any, TokenType.SOME: exp.Any,
} }
RESERVED_KEYWORDS = { RESERVED_TOKENS = {
*Tokenizer.SINGLE_TOKENS.values(), *Tokenizer.SINGLE_TOKENS.values(),
TokenType.SELECT, TokenType.SELECT,
} }
@ -245,9 +270,11 @@ class Parser(metaclass=_Parser):
CREATABLES = { CREATABLES = {
TokenType.COLUMN, TokenType.COLUMN,
TokenType.CONSTRAINT,
TokenType.FUNCTION, TokenType.FUNCTION,
TokenType.INDEX, TokenType.INDEX,
TokenType.PROCEDURE, TokenType.PROCEDURE,
TokenType.FOREIGN_KEY,
*DB_CREATABLES, *DB_CREATABLES,
} }
@ -291,6 +318,7 @@ class Parser(metaclass=_Parser):
TokenType.NATURAL, TokenType.NATURAL,
TokenType.NEXT, TokenType.NEXT,
TokenType.OFFSET, TokenType.OFFSET,
TokenType.OPERATOR,
TokenType.ORDINALITY, TokenType.ORDINALITY,
TokenType.OVERLAPS, TokenType.OVERLAPS,
TokenType.OVERWRITE, TokenType.OVERWRITE,
@ -299,7 +327,10 @@ class Parser(metaclass=_Parser):
TokenType.PIVOT, TokenType.PIVOT,
TokenType.PRAGMA, TokenType.PRAGMA,
TokenType.RANGE, TokenType.RANGE,
TokenType.RECURSIVE,
TokenType.REFERENCES, TokenType.REFERENCES,
TokenType.REFRESH,
TokenType.REPLACE,
TokenType.RIGHT, TokenType.RIGHT,
TokenType.ROW, TokenType.ROW,
TokenType.ROWS, TokenType.ROWS,
@ -390,6 +421,7 @@ class Parser(metaclass=_Parser):
} }
EQUALITY = { EQUALITY = {
TokenType.COLON_EQ: exp.PropertyEQ,
TokenType.EQ: exp.EQ, TokenType.EQ: exp.EQ,
TokenType.NEQ: exp.NEQ, TokenType.NEQ: exp.NEQ,
TokenType.NULLSAFE_EQ: exp.NullSafeEQ, TokenType.NULLSAFE_EQ: exp.NullSafeEQ,
@ -406,7 +438,6 @@ class Parser(metaclass=_Parser):
TokenType.AMP: exp.BitwiseAnd, TokenType.AMP: exp.BitwiseAnd,
TokenType.CARET: exp.BitwiseXor, TokenType.CARET: exp.BitwiseXor,
TokenType.PIPE: exp.BitwiseOr, TokenType.PIPE: exp.BitwiseOr,
TokenType.DPIPE: exp.DPipe,
} }
TERM = { TERM = {
@ -423,6 +454,8 @@ class Parser(metaclass=_Parser):
TokenType.STAR: exp.Mul, TokenType.STAR: exp.Mul,
} }
EXPONENT: t.Dict[TokenType, t.Type[exp.Expression]] = {}
TIMES = { TIMES = {
TokenType.TIME, TokenType.TIME,
TokenType.TIMETZ, TokenType.TIMETZ,
@ -558,6 +591,7 @@ class Parser(metaclass=_Parser):
TokenType.MERGE: lambda self: self._parse_merge(), TokenType.MERGE: lambda self: self._parse_merge(),
TokenType.PIVOT: lambda self: self._parse_simplified_pivot(), TokenType.PIVOT: lambda self: self._parse_simplified_pivot(),
TokenType.PRAGMA: lambda self: self.expression(exp.Pragma, this=self._parse_expression()), TokenType.PRAGMA: lambda self: self.expression(exp.Pragma, this=self._parse_expression()),
TokenType.REFRESH: lambda self: self._parse_refresh(),
TokenType.ROLLBACK: lambda self: self._parse_commit_or_rollback(), TokenType.ROLLBACK: lambda self: self._parse_commit_or_rollback(),
TokenType.SET: lambda self: self._parse_set(), TokenType.SET: lambda self: self._parse_set(),
TokenType.UNCACHE: lambda self: self._parse_uncache(), TokenType.UNCACHE: lambda self: self._parse_uncache(),
@ -697,6 +731,7 @@ class Parser(metaclass=_Parser):
exp.StabilityProperty, this=exp.Literal.string("STABLE") exp.StabilityProperty, this=exp.Literal.string("STABLE")
), ),
"STORED": lambda self: self._parse_stored(), "STORED": lambda self: self._parse_stored(),
"SYSTEM_VERSIONING": lambda self: self._parse_system_versioning_property(),
"TBLPROPERTIES": lambda self: self._parse_wrapped_csv(self._parse_property), "TBLPROPERTIES": lambda self: self._parse_wrapped_csv(self._parse_property),
"TEMP": lambda self: self.expression(exp.TemporaryProperty), "TEMP": lambda self: self.expression(exp.TemporaryProperty),
"TEMPORARY": lambda self: self.expression(exp.TemporaryProperty), "TEMPORARY": lambda self: self.expression(exp.TemporaryProperty),
@ -754,6 +789,7 @@ class Parser(metaclass=_Parser):
) )
or self.expression(exp.OnProperty, this=self._parse_id_var()), or self.expression(exp.OnProperty, this=self._parse_id_var()),
"PATH": lambda self: self.expression(exp.PathColumnConstraint, this=self._parse_string()), "PATH": lambda self: self.expression(exp.PathColumnConstraint, this=self._parse_string()),
"PERIOD": lambda self: self._parse_period_for_system_time(),
"PRIMARY KEY": lambda self: self._parse_primary_key(), "PRIMARY KEY": lambda self: self._parse_primary_key(),
"REFERENCES": lambda self: self._parse_references(match=False), "REFERENCES": lambda self: self._parse_references(match=False),
"TITLE": lambda self: self.expression( "TITLE": lambda self: self.expression(
@ -775,7 +811,7 @@ class Parser(metaclass=_Parser):
"RENAME": lambda self: self._parse_alter_table_rename(), "RENAME": lambda self: self._parse_alter_table_rename(),
} }
SCHEMA_UNNAMED_CONSTRAINTS = {"CHECK", "FOREIGN KEY", "LIKE", "PRIMARY KEY", "UNIQUE"} SCHEMA_UNNAMED_CONSTRAINTS = {"CHECK", "FOREIGN KEY", "LIKE", "PRIMARY KEY", "UNIQUE", "PERIOD"}
NO_PAREN_FUNCTION_PARSERS = { NO_PAREN_FUNCTION_PARSERS = {
"ANY": lambda self: self.expression(exp.Any, this=self._parse_bitwise()), "ANY": lambda self: self.expression(exp.Any, this=self._parse_bitwise()),
@ -794,14 +830,11 @@ class Parser(metaclass=_Parser):
FUNCTION_PARSERS = { FUNCTION_PARSERS = {
"ANY_VALUE": lambda self: self._parse_any_value(), "ANY_VALUE": lambda self: self._parse_any_value(),
"CAST": lambda self: self._parse_cast(self.STRICT_CAST), "CAST": lambda self: self._parse_cast(self.STRICT_CAST),
"CONCAT": lambda self: self._parse_concat(),
"CONCAT_WS": lambda self: self._parse_concat_ws(),
"CONVERT": lambda self: self._parse_convert(self.STRICT_CAST), "CONVERT": lambda self: self._parse_convert(self.STRICT_CAST),
"DECODE": lambda self: self._parse_decode(), "DECODE": lambda self: self._parse_decode(),
"EXTRACT": lambda self: self._parse_extract(), "EXTRACT": lambda self: self._parse_extract(),
"JSON_OBJECT": lambda self: self._parse_json_object(), "JSON_OBJECT": lambda self: self._parse_json_object(),
"JSON_TABLE": lambda self: self._parse_json_table(), "JSON_TABLE": lambda self: self._parse_json_table(),
"LOG": lambda self: self._parse_logarithm(),
"MATCH": lambda self: self._parse_match_against(), "MATCH": lambda self: self._parse_match_against(),
"OPENJSON": lambda self: self._parse_open_json(), "OPENJSON": lambda self: self._parse_open_json(),
"POSITION": lambda self: self._parse_position(), "POSITION": lambda self: self._parse_position(),
@ -877,6 +910,7 @@ class Parser(metaclass=_Parser):
CLONE_KINDS = {"TIMESTAMP", "OFFSET", "STATEMENT"} CLONE_KINDS = {"TIMESTAMP", "OFFSET", "STATEMENT"}
OPCLASS_FOLLOW_KEYWORDS = {"ASC", "DESC", "NULLS"} OPCLASS_FOLLOW_KEYWORDS = {"ASC", "DESC", "NULLS"}
OPTYPE_FOLLOW_TOKENS = {TokenType.COMMA, TokenType.R_PAREN}
TABLE_INDEX_HINT_TOKENS = {TokenType.FORCE, TokenType.IGNORE, TokenType.USE} TABLE_INDEX_HINT_TOKENS = {TokenType.FORCE, TokenType.IGNORE, TokenType.USE}
@ -896,17 +930,13 @@ class Parser(metaclass=_Parser):
STRICT_CAST = True STRICT_CAST = True
# A NULL arg in CONCAT yields NULL by default
CONCAT_NULL_OUTPUTS_STRING = False
PREFIXED_PIVOT_COLUMNS = False PREFIXED_PIVOT_COLUMNS = False
IDENTIFY_PIVOT_STRINGS = False IDENTIFY_PIVOT_STRINGS = False
LOG_BASE_FIRST = True
LOG_DEFAULTS_TO_LN = False LOG_DEFAULTS_TO_LN = False
# Whether or not ADD is present for each column added by ALTER TABLE # Whether or not ADD is present for each column added by ALTER TABLE
ALTER_TABLE_ADD_COLUMN_KEYWORD = True ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = True
# Whether or not the table sample clause expects CSV syntax # Whether or not the table sample clause expects CSV syntax
TABLESAMPLE_CSV = False TABLESAMPLE_CSV = False
@ -921,6 +951,7 @@ class Parser(metaclass=_Parser):
"error_level", "error_level",
"error_message_context", "error_message_context",
"max_errors", "max_errors",
"dialect",
"sql", "sql",
"errors", "errors",
"_tokens", "_tokens",
@ -929,35 +960,25 @@ class Parser(metaclass=_Parser):
"_next", "_next",
"_prev", "_prev",
"_prev_comments", "_prev_comments",
"_tokenizer",
) )
# Autofilled # Autofilled
TOKENIZER_CLASS: t.Type[Tokenizer] = Tokenizer
INDEX_OFFSET: int = 0
UNNEST_COLUMN_ONLY: bool = False
ALIAS_POST_TABLESAMPLE: bool = False
STRICT_STRING_CONCAT = False
SUPPORTS_USER_DEFINED_TYPES = True
NORMALIZE_FUNCTIONS = "upper"
NULL_ORDERING: str = "nulls_are_small"
SHOW_TRIE: t.Dict = {} SHOW_TRIE: t.Dict = {}
SET_TRIE: t.Dict = {} SET_TRIE: t.Dict = {}
FORMAT_MAPPING: t.Dict[str, str] = {}
FORMAT_TRIE: t.Dict = {}
TIME_MAPPING: t.Dict[str, str] = {}
TIME_TRIE: t.Dict = {}
def __init__( def __init__(
self, self,
error_level: t.Optional[ErrorLevel] = None, error_level: t.Optional[ErrorLevel] = None,
error_message_context: int = 100, error_message_context: int = 100,
max_errors: int = 3, max_errors: int = 3,
dialect: DialectType = None,
): ):
from sqlglot.dialects import Dialect
self.error_level = error_level or ErrorLevel.IMMEDIATE self.error_level = error_level or ErrorLevel.IMMEDIATE
self.error_message_context = error_message_context self.error_message_context = error_message_context
self.max_errors = max_errors self.max_errors = max_errors
self._tokenizer = self.TOKENIZER_CLASS() self.dialect = Dialect.get_or_raise(dialect)
self.reset() self.reset()
def reset(self): def reset(self):
@ -1384,7 +1405,7 @@ class Parser(metaclass=_Parser):
if self._match_texts(self.CLONE_KEYWORDS): if self._match_texts(self.CLONE_KEYWORDS):
copy = self._prev.text.lower() == "copy" copy = self._prev.text.lower() == "copy"
clone = self._parse_table(schema=True) clone = self._parse_table(schema=True)
when = self._match_texts({"AT", "BEFORE"}) and self._prev.text.upper() when = self._match_texts(("AT", "BEFORE")) and self._prev.text.upper()
clone_kind = ( clone_kind = (
self._match(TokenType.L_PAREN) self._match(TokenType.L_PAREN)
and self._match_texts(self.CLONE_KINDS) and self._match_texts(self.CLONE_KINDS)
@ -1524,6 +1545,22 @@ class Parser(metaclass=_Parser):
return self.expression(exp.StabilityProperty, this=exp.Literal.string("VOLATILE")) return self.expression(exp.StabilityProperty, this=exp.Literal.string("VOLATILE"))
def _parse_system_versioning_property(self) -> exp.WithSystemVersioningProperty:
self._match_pair(TokenType.EQ, TokenType.ON)
prop = self.expression(exp.WithSystemVersioningProperty)
if self._match(TokenType.L_PAREN):
self._match_text_seq("HISTORY_TABLE", "=")
prop.set("this", self._parse_table_parts())
if self._match(TokenType.COMMA):
self._match_text_seq("DATA_CONSISTENCY_CHECK", "=")
prop.set("expression", self._advance_any() and self._prev.text.upper())
self._match_r_paren()
return prop
def _parse_with_property( def _parse_with_property(
self, self,
) -> t.Optional[exp.Expression] | t.List[exp.Expression]: ) -> t.Optional[exp.Expression] | t.List[exp.Expression]:
@ -2140,7 +2177,11 @@ class Parser(metaclass=_Parser):
return self._parse_expressions() return self._parse_expressions()
def _parse_select( def _parse_select(
self, nested: bool = False, table: bool = False, parse_subquery_alias: bool = True self,
nested: bool = False,
table: bool = False,
parse_subquery_alias: bool = True,
parse_set_operation: bool = True,
) -> t.Optional[exp.Expression]: ) -> t.Optional[exp.Expression]:
cte = self._parse_with() cte = self._parse_with()
@ -2216,7 +2257,11 @@ class Parser(metaclass=_Parser):
t.cast(exp.From, self._parse_from(skip_from_token=True)) t.cast(exp.From, self._parse_from(skip_from_token=True))
) )
else: else:
this = self._parse_table() if table else self._parse_select(nested=True) this = (
self._parse_table()
if table
else self._parse_select(nested=True, parse_set_operation=False)
)
this = self._parse_set_operations(self._parse_query_modifiers(this)) this = self._parse_set_operations(self._parse_query_modifiers(this))
self._match_r_paren() self._match_r_paren()
@ -2235,7 +2280,9 @@ class Parser(metaclass=_Parser):
else: else:
this = None this = None
if parse_set_operation:
return self._parse_set_operations(this) return self._parse_set_operations(this)
return this
def _parse_with(self, skip_with_token: bool = False) -> t.Optional[exp.With]: def _parse_with(self, skip_with_token: bool = False) -> t.Optional[exp.With]:
if not skip_with_token and not self._match(TokenType.WITH): if not skip_with_token and not self._match(TokenType.WITH):
@ -2563,9 +2610,8 @@ class Parser(metaclass=_Parser):
if self._match_texts(self.OPCLASS_FOLLOW_KEYWORDS, advance=False): if self._match_texts(self.OPCLASS_FOLLOW_KEYWORDS, advance=False):
return this return this
opclass = self._parse_var(any_token=True) if not self._match_set(self.OPTYPE_FOLLOW_TOKENS, advance=False):
if opclass: return self.expression(exp.Opclass, this=this, expression=self._parse_table_parts())
return self.expression(exp.Opclass, this=this, expression=opclass)
return this return this
@ -2630,7 +2676,7 @@ class Parser(metaclass=_Parser):
while self._match_set(self.TABLE_INDEX_HINT_TOKENS): while self._match_set(self.TABLE_INDEX_HINT_TOKENS):
hint = exp.IndexTableHint(this=self._prev.text.upper()) hint = exp.IndexTableHint(this=self._prev.text.upper())
self._match_texts({"INDEX", "KEY"}) self._match_texts(("INDEX", "KEY"))
if self._match(TokenType.FOR): if self._match(TokenType.FOR):
hint.set("target", self._advance_any() and self._prev.text.upper()) hint.set("target", self._advance_any() and self._prev.text.upper())
@ -2650,7 +2696,7 @@ class Parser(metaclass=_Parser):
def _parse_table_parts(self, schema: bool = False) -> exp.Table: def _parse_table_parts(self, schema: bool = False) -> exp.Table:
catalog = None catalog = None
db = None db = None
table = self._parse_table_part(schema=schema) table: t.Optional[exp.Expression | str] = self._parse_table_part(schema=schema)
while self._match(TokenType.DOT): while self._match(TokenType.DOT):
if catalog: if catalog:
@ -2661,7 +2707,7 @@ class Parser(metaclass=_Parser):
else: else:
catalog = db catalog = db
db = table db = table
table = self._parse_table_part(schema=schema) table = self._parse_table_part(schema=schema) or ""
if not table: if not table:
self.raise_error(f"Expected table name but got {self._curr}") self.raise_error(f"Expected table name but got {self._curr}")
@ -2709,7 +2755,7 @@ class Parser(metaclass=_Parser):
if version: if version:
this.set("version", version) this.set("version", version)
if self.ALIAS_POST_TABLESAMPLE: if self.dialect.ALIAS_POST_TABLESAMPLE:
table_sample = self._parse_table_sample() table_sample = self._parse_table_sample()
alias = self._parse_table_alias(alias_tokens=alias_tokens or self.TABLE_ALIAS_TOKENS) alias = self._parse_table_alias(alias_tokens=alias_tokens or self.TABLE_ALIAS_TOKENS)
@ -2724,7 +2770,7 @@ class Parser(metaclass=_Parser):
if not this.args.get("pivots"): if not this.args.get("pivots"):
this.set("pivots", self._parse_pivots()) this.set("pivots", self._parse_pivots())
if not self.ALIAS_POST_TABLESAMPLE: if not self.dialect.ALIAS_POST_TABLESAMPLE:
table_sample = self._parse_table_sample() table_sample = self._parse_table_sample()
if table_sample: if table_sample:
@ -2776,13 +2822,13 @@ class Parser(metaclass=_Parser):
if not self._match(TokenType.UNNEST): if not self._match(TokenType.UNNEST):
return None return None
expressions = self._parse_wrapped_csv(self._parse_type) expressions = self._parse_wrapped_csv(self._parse_equality)
offset = self._match_pair(TokenType.WITH, TokenType.ORDINALITY) offset = self._match_pair(TokenType.WITH, TokenType.ORDINALITY)
alias = self._parse_table_alias() if with_alias else None alias = self._parse_table_alias() if with_alias else None
if alias: if alias:
if self.UNNEST_COLUMN_ONLY: if self.dialect.UNNEST_COLUMN_ONLY:
if alias.args.get("columns"): if alias.args.get("columns"):
self.raise_error("Unexpected extra column alias in unnest.") self.raise_error("Unexpected extra column alias in unnest.")
@ -2845,7 +2891,7 @@ class Parser(metaclass=_Parser):
num = ( num = (
self._parse_factor() self._parse_factor()
if self._match(TokenType.NUMBER, advance=False) if self._match(TokenType.NUMBER, advance=False)
else self._parse_primary() else self._parse_primary() or self._parse_placeholder()
) )
if self._match_text_seq("BUCKET"): if self._match_text_seq("BUCKET"):
@ -3108,10 +3154,10 @@ class Parser(metaclass=_Parser):
if ( if (
not explicitly_null_ordered not explicitly_null_ordered
and ( and (
(not desc and self.NULL_ORDERING == "nulls_are_small") (not desc and self.dialect.NULL_ORDERING == "nulls_are_small")
or (desc and self.NULL_ORDERING != "nulls_are_small") or (desc and self.dialect.NULL_ORDERING != "nulls_are_small")
) )
and self.NULL_ORDERING != "nulls_are_last" and self.dialect.NULL_ORDERING != "nulls_are_last"
): ):
nulls_first = True nulls_first = True
@ -3124,7 +3170,7 @@ class Parser(metaclass=_Parser):
comments = self._prev_comments comments = self._prev_comments
if top: if top:
limit_paren = self._match(TokenType.L_PAREN) limit_paren = self._match(TokenType.L_PAREN)
expression = self._parse_number() expression = self._parse_term() if limit_paren else self._parse_number()
if limit_paren: if limit_paren:
self._match_r_paren() self._match_r_paren()
@ -3225,7 +3271,9 @@ class Parser(metaclass=_Parser):
this=this, this=this,
distinct=self._match(TokenType.DISTINCT) or not self._match(TokenType.ALL), distinct=self._match(TokenType.DISTINCT) or not self._match(TokenType.ALL),
by_name=self._match_text_seq("BY", "NAME"), by_name=self._match_text_seq("BY", "NAME"),
expression=self._parse_set_operations(self._parse_select(nested=True)), expression=self._parse_set_operations(
self._parse_select(nested=True, parse_set_operation=False)
),
) )
def _parse_expression(self) -> t.Optional[exp.Expression]: def _parse_expression(self) -> t.Optional[exp.Expression]:
@ -3287,7 +3335,8 @@ class Parser(metaclass=_Parser):
unnest = self._parse_unnest(with_alias=False) unnest = self._parse_unnest(with_alias=False)
if unnest: if unnest:
this = self.expression(exp.In, this=this, unnest=unnest) this = self.expression(exp.In, this=this, unnest=unnest)
elif self._match(TokenType.L_PAREN): elif self._match_set((TokenType.L_PAREN, TokenType.L_BRACKET)):
matched_l_paren = self._prev.token_type == TokenType.L_PAREN
expressions = self._parse_csv(lambda: self._parse_select_or_expression(alias=alias)) expressions = self._parse_csv(lambda: self._parse_select_or_expression(alias=alias))
if len(expressions) == 1 and isinstance(expressions[0], exp.Subqueryable): if len(expressions) == 1 and isinstance(expressions[0], exp.Subqueryable):
@ -3295,13 +3344,16 @@ class Parser(metaclass=_Parser):
else: else:
this = self.expression(exp.In, this=this, expressions=expressions) this = self.expression(exp.In, this=this, expressions=expressions)
if matched_l_paren:
self._match_r_paren(this) self._match_r_paren(this)
elif not self._match(TokenType.R_BRACKET, expression=this):
self.raise_error("Expecting ]")
else: else:
this = self.expression(exp.In, this=this, field=self._parse_field()) this = self.expression(exp.In, this=this, field=self._parse_field())
return this return this
def _parse_between(self, this: exp.Expression) -> exp.Between: def _parse_between(self, this: t.Optional[exp.Expression]) -> exp.Between:
low = self._parse_bitwise() low = self._parse_bitwise()
self._match(TokenType.AND) self._match(TokenType.AND)
high = self._parse_bitwise() high = self._parse_bitwise()
@ -3357,6 +3409,13 @@ class Parser(metaclass=_Parser):
this=this, this=this,
expression=self._parse_term(), expression=self._parse_term(),
) )
elif self.dialect.DPIPE_IS_STRING_CONCAT and self._match(TokenType.DPIPE):
this = self.expression(
exp.DPipe,
this=this,
expression=self._parse_term(),
safe=not self.dialect.STRICT_STRING_CONCAT,
)
elif self._match(TokenType.DQMARK): elif self._match(TokenType.DQMARK):
this = self.expression(exp.Coalesce, this=this, expressions=self._parse_term()) this = self.expression(exp.Coalesce, this=this, expressions=self._parse_term())
elif self._match_pair(TokenType.LT, TokenType.LT): elif self._match_pair(TokenType.LT, TokenType.LT):
@ -3376,7 +3435,17 @@ class Parser(metaclass=_Parser):
return self._parse_tokens(self._parse_factor, self.TERM) return self._parse_tokens(self._parse_factor, self.TERM)
def _parse_factor(self) -> t.Optional[exp.Expression]: def _parse_factor(self) -> t.Optional[exp.Expression]:
return self._parse_tokens(self._parse_unary, self.FACTOR) if self.EXPONENT:
factor = self._parse_tokens(self._parse_exponent, self.FACTOR)
else:
factor = self._parse_tokens(self._parse_unary, self.FACTOR)
if isinstance(factor, exp.Div):
factor.args["typed"] = self.dialect.TYPED_DIVISION
factor.args["safe"] = self.dialect.SAFE_DIVISION
return factor
def _parse_exponent(self) -> t.Optional[exp.Expression]:
return self._parse_tokens(self._parse_unary, self.EXPONENT)
def _parse_unary(self) -> t.Optional[exp.Expression]: def _parse_unary(self) -> t.Optional[exp.Expression]:
if self._match_set(self.UNARY_PARSERS): if self._match_set(self.UNARY_PARSERS):
@ -3427,14 +3496,14 @@ class Parser(metaclass=_Parser):
) )
if identifier: if identifier:
tokens = self._tokenizer.tokenize(identifier.name) tokens = self.dialect.tokenize(identifier.name)
if len(tokens) != 1: if len(tokens) != 1:
self.raise_error("Unexpected identifier", self._prev) self.raise_error("Unexpected identifier", self._prev)
if tokens[0].token_type in self.TYPE_TOKENS: if tokens[0].token_type in self.TYPE_TOKENS:
self._prev = tokens[0] self._prev = tokens[0]
elif self.SUPPORTS_USER_DEFINED_TYPES: elif self.dialect.SUPPORTS_USER_DEFINED_TYPES:
type_name = identifier.name type_name = identifier.name
while self._match(TokenType.DOT): while self._match(TokenType.DOT):
@ -3713,6 +3782,7 @@ class Parser(metaclass=_Parser):
if not self._curr: if not self._curr:
return None return None
comments = self._curr.comments
token_type = self._curr.token_type token_type = self._curr.token_type
this = self._curr.text this = self._curr.text
upper = this.upper() upper = this.upper()
@ -3754,13 +3824,22 @@ class Parser(metaclass=_Parser):
args = self._parse_csv(lambda: self._parse_lambda(alias=alias)) args = self._parse_csv(lambda: self._parse_lambda(alias=alias))
if function and not anonymous: if function and not anonymous:
func = self.validate_expression(function(args), args) if "dialect" in function.__code__.co_varnames:
if not self.NORMALIZE_FUNCTIONS: func = function(args, dialect=self.dialect)
else:
func = function(args)
func = self.validate_expression(func, args)
if not self.dialect.NORMALIZE_FUNCTIONS:
func.meta["name"] = this func.meta["name"] = this
this = func this = func
else: else:
this = self.expression(exp.Anonymous, this=this, expressions=args) this = self.expression(exp.Anonymous, this=this, expressions=args)
if isinstance(this, exp.Expression):
this.add_comments(comments)
self._match_r_paren(this) self._match_r_paren(this)
return self._parse_window(this) return self._parse_window(this)
@ -3875,6 +3954,11 @@ class Parser(metaclass=_Parser):
not_null=self._match_pair(TokenType.NOT, TokenType.NULL), not_null=self._match_pair(TokenType.NOT, TokenType.NULL),
) )
) )
elif kind and self._match_pair(TokenType.ALIAS, TokenType.L_PAREN, advance=False):
self._match(TokenType.ALIAS)
constraints.append(
self.expression(exp.TransformColumnConstraint, this=self._parse_field())
)
while True: while True:
constraint = self._parse_column_constraint() constraint = self._parse_column_constraint()
@ -3917,7 +4001,11 @@ class Parser(metaclass=_Parser):
def _parse_generated_as_identity( def _parse_generated_as_identity(
self, self,
) -> exp.GeneratedAsIdentityColumnConstraint | exp.ComputedColumnConstraint: ) -> (
exp.GeneratedAsIdentityColumnConstraint
| exp.ComputedColumnConstraint
| exp.GeneratedAsRowColumnConstraint
):
if self._match_text_seq("BY", "DEFAULT"): if self._match_text_seq("BY", "DEFAULT"):
on_null = self._match_pair(TokenType.ON, TokenType.NULL) on_null = self._match_pair(TokenType.ON, TokenType.NULL)
this = self.expression( this = self.expression(
@ -3928,6 +4016,14 @@ class Parser(metaclass=_Parser):
this = self.expression(exp.GeneratedAsIdentityColumnConstraint, this=True) this = self.expression(exp.GeneratedAsIdentityColumnConstraint, this=True)
self._match(TokenType.ALIAS) self._match(TokenType.ALIAS)
if self._match_text_seq("ROW"):
start = self._match_text_seq("START")
if not start:
self._match(TokenType.END)
hidden = self._match_text_seq("HIDDEN")
return self.expression(exp.GeneratedAsRowColumnConstraint, start=start, hidden=hidden)
identity = self._match_text_seq("IDENTITY") identity = self._match_text_seq("IDENTITY")
if self._match(TokenType.L_PAREN): if self._match(TokenType.L_PAREN):
@ -4100,6 +4196,16 @@ class Parser(metaclass=_Parser):
def _parse_primary_key_part(self) -> t.Optional[exp.Expression]: def _parse_primary_key_part(self) -> t.Optional[exp.Expression]:
return self._parse_field() return self._parse_field()
def _parse_period_for_system_time(self) -> exp.PeriodForSystemTimeConstraint:
self._match(TokenType.TIMESTAMP_SNAPSHOT)
id_vars = self._parse_wrapped_id_vars()
return self.expression(
exp.PeriodForSystemTimeConstraint,
this=seq_get(id_vars, 0),
expression=seq_get(id_vars, 1),
)
def _parse_primary_key( def _parse_primary_key(
self, wrapped_optional: bool = False, in_props: bool = False self, wrapped_optional: bool = False, in_props: bool = False
) -> exp.PrimaryKeyColumnConstraint | exp.PrimaryKey: ) -> exp.PrimaryKeyColumnConstraint | exp.PrimaryKey:
@ -4145,7 +4251,7 @@ class Parser(metaclass=_Parser):
elif not this or this.name.upper() == "ARRAY": elif not this or this.name.upper() == "ARRAY":
this = self.expression(exp.Array, expressions=expressions) this = self.expression(exp.Array, expressions=expressions)
else: else:
expressions = apply_index_offset(this, expressions, -self.INDEX_OFFSET) expressions = apply_index_offset(this, expressions, -self.dialect.INDEX_OFFSET)
this = self.expression(exp.Bracket, this=this, expressions=expressions) this = self.expression(exp.Bracket, this=this, expressions=expressions)
self._add_comments(this) self._add_comments(this)
@ -4259,8 +4365,8 @@ class Parser(metaclass=_Parser):
format=exp.Literal.string( format=exp.Literal.string(
format_time( format_time(
fmt_string.this if fmt_string else "", fmt_string.this if fmt_string else "",
self.FORMAT_MAPPING or self.TIME_MAPPING, self.dialect.FORMAT_MAPPING or self.dialect.TIME_MAPPING,
self.FORMAT_TRIE or self.TIME_TRIE, self.dialect.FORMAT_TRIE or self.dialect.TIME_TRIE,
) )
), ),
) )
@ -4280,30 +4386,6 @@ class Parser(metaclass=_Parser):
exp.Cast if strict else exp.TryCast, this=this, to=to, format=fmt, safe=safe exp.Cast if strict else exp.TryCast, this=this, to=to, format=fmt, safe=safe
) )
def _parse_concat(self) -> t.Optional[exp.Expression]:
args = self._parse_csv(self._parse_conjunction)
if self.CONCAT_NULL_OUTPUTS_STRING:
args = self._ensure_string_if_null(args)
# Some dialects (e.g. Trino) don't allow a single-argument CONCAT call, so when
# we find such a call we replace it with its argument.
if len(args) == 1:
return args[0]
return self.expression(
exp.Concat if self.STRICT_STRING_CONCAT else exp.SafeConcat, expressions=args
)
def _parse_concat_ws(self) -> t.Optional[exp.Expression]:
args = self._parse_csv(self._parse_conjunction)
if len(args) < 2:
return self.expression(exp.ConcatWs, expressions=args)
delim, *values = args
if self.CONCAT_NULL_OUTPUTS_STRING:
values = self._ensure_string_if_null(values)
return self.expression(exp.ConcatWs, expressions=[delim] + values)
def _parse_string_agg(self) -> exp.Expression: def _parse_string_agg(self) -> exp.Expression:
if self._match(TokenType.DISTINCT): if self._match(TokenType.DISTINCT):
args: t.List[t.Optional[exp.Expression]] = [ args: t.List[t.Optional[exp.Expression]] = [
@ -4495,19 +4577,6 @@ class Parser(metaclass=_Parser):
empty_handling=empty_handling, empty_handling=empty_handling,
) )
def _parse_logarithm(self) -> exp.Func:
# Default argument order is base, expression
args = self._parse_csv(self._parse_range)
if len(args) > 1:
if not self.LOG_BASE_FIRST:
args.reverse()
return exp.Log.from_arg_list(args)
return self.expression(
exp.Ln if self.LOG_DEFAULTS_TO_LN else exp.Log, this=seq_get(args, 0)
)
def _parse_match_against(self) -> exp.MatchAgainst: def _parse_match_against(self) -> exp.MatchAgainst:
expressions = self._parse_csv(self._parse_column) expressions = self._parse_csv(self._parse_column)
@ -4755,6 +4824,7 @@ class Parser(metaclass=_Parser):
self, this: t.Optional[exp.Expression], explicit: bool = False self, this: t.Optional[exp.Expression], explicit: bool = False
) -> t.Optional[exp.Expression]: ) -> t.Optional[exp.Expression]:
any_token = self._match(TokenType.ALIAS) any_token = self._match(TokenType.ALIAS)
comments = self._prev_comments
if explicit and not any_token: if explicit and not any_token:
return this return this
@ -4762,6 +4832,7 @@ class Parser(metaclass=_Parser):
if self._match(TokenType.L_PAREN): if self._match(TokenType.L_PAREN):
aliases = self.expression( aliases = self.expression(
exp.Aliases, exp.Aliases,
comments=comments,
this=this, this=this,
expressions=self._parse_csv(lambda: self._parse_id_var(any_token)), expressions=self._parse_csv(lambda: self._parse_id_var(any_token)),
) )
@ -4771,7 +4842,7 @@ class Parser(metaclass=_Parser):
alias = self._parse_id_var(any_token) alias = self._parse_id_var(any_token)
if alias: if alias:
return self.expression(exp.Alias, this=this, alias=alias) return self.expression(exp.Alias, comments=comments, this=this, alias=alias)
return this return this
@ -4792,8 +4863,8 @@ class Parser(metaclass=_Parser):
return None return None
def _parse_string(self) -> t.Optional[exp.Expression]: def _parse_string(self) -> t.Optional[exp.Expression]:
if self._match(TokenType.STRING): if self._match_set((TokenType.STRING, TokenType.RAW_STRING)):
return self.PRIMARY_PARSERS[TokenType.STRING](self, self._prev) return self.PRIMARY_PARSERS[self._prev.token_type](self, self._prev)
return self._parse_placeholder() return self._parse_placeholder()
def _parse_string_as_identifier(self) -> t.Optional[exp.Identifier]: def _parse_string_as_identifier(self) -> t.Optional[exp.Identifier]:
@ -4821,7 +4892,7 @@ class Parser(metaclass=_Parser):
return self._parse_placeholder() return self._parse_placeholder()
def _advance_any(self) -> t.Optional[Token]: def _advance_any(self) -> t.Optional[Token]:
if self._curr and self._curr.token_type not in self.RESERVED_KEYWORDS: if self._curr and self._curr.token_type not in self.RESERVED_TOKENS:
self._advance() self._advance()
return self._prev return self._prev
return None return None
@ -4951,7 +5022,7 @@ class Parser(metaclass=_Parser):
if self._match_texts(self.TRANSACTION_KIND): if self._match_texts(self.TRANSACTION_KIND):
this = self._prev.text this = self._prev.text
self._match_texts({"TRANSACTION", "WORK"}) self._match_texts(("TRANSACTION", "WORK"))
modes = [] modes = []
while True: while True:
@ -4971,7 +5042,7 @@ class Parser(metaclass=_Parser):
savepoint = None savepoint = None
is_rollback = self._prev.token_type == TokenType.ROLLBACK is_rollback = self._prev.token_type == TokenType.ROLLBACK
self._match_texts({"TRANSACTION", "WORK"}) self._match_texts(("TRANSACTION", "WORK"))
if self._match_text_seq("TO"): if self._match_text_seq("TO"):
self._match_text_seq("SAVEPOINT") self._match_text_seq("SAVEPOINT")
@ -4986,6 +5057,10 @@ class Parser(metaclass=_Parser):
return self.expression(exp.Commit, chain=chain) return self.expression(exp.Commit, chain=chain)
def _parse_refresh(self) -> exp.Refresh:
self._match(TokenType.TABLE)
return self.expression(exp.Refresh, this=self._parse_string() or self._parse_table())
def _parse_add_column(self) -> t.Optional[exp.Expression]: def _parse_add_column(self) -> t.Optional[exp.Expression]:
if not self._match_text_seq("ADD"): if not self._match_text_seq("ADD"):
return None return None
@ -5050,10 +5125,9 @@ class Parser(metaclass=_Parser):
return self._parse_csv(self._parse_add_constraint) return self._parse_csv(self._parse_add_constraint)
self._retreat(index) self._retreat(index)
if not self.ALTER_TABLE_ADD_COLUMN_KEYWORD and self._match_text_seq("ADD"): if not self.ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN and self._match_text_seq("ADD"):
return self._parse_csv(self._parse_field_def) return self._parse_wrapped_csv(self._parse_field_def, optional=True)
return self._parse_wrapped_csv(self._parse_add_column, optional=True)
return self._parse_csv(self._parse_add_column)
def _parse_alter_table_alter(self) -> exp.AlterColumn: def _parse_alter_table_alter(self) -> exp.AlterColumn:
self._match(TokenType.COLUMN) self._match(TokenType.COLUMN)
@ -5198,7 +5272,7 @@ class Parser(metaclass=_Parser):
) -> t.Optional[exp.Expression]: ) -> t.Optional[exp.Expression]:
index = self._index index = self._index
if kind in {"GLOBAL", "SESSION"} and self._match_text_seq("TRANSACTION"): if kind in ("GLOBAL", "SESSION") and self._match_text_seq("TRANSACTION"):
return self._parse_set_transaction(global_=kind == "GLOBAL") return self._parse_set_transaction(global_=kind == "GLOBAL")
left = self._parse_primary() or self._parse_id_var() left = self._parse_primary() or self._parse_id_var()
@ -5292,7 +5366,9 @@ class Parser(metaclass=_Parser):
self._match_r_paren() self._match_r_paren()
return self.expression(exp.DictRange, this=this, min=min, max=max) return self.expression(exp.DictRange, this=this, min=min, max=max)
def _parse_comprehension(self, this: exp.Expression) -> t.Optional[exp.Comprehension]: def _parse_comprehension(
self, this: t.Optional[exp.Expression]
) -> t.Optional[exp.Comprehension]:
index = self._index index = self._index
expression = self._parse_column() expression = self._parse_column()
if not self._match(TokenType.IN): if not self._match(TokenType.IN):
@ -5441,10 +5517,3 @@ class Parser(metaclass=_Parser):
else: else:
column.replace(dot_or_id) column.replace(dot_or_id)
return node return node
def _ensure_string_if_null(self, values: t.List[exp.Expression]) -> t.List[exp.Expression]:
return [
exp.func("COALESCE", exp.cast(value, "text"), exp.Literal.string(""))
for value in values
if value
]

40
sqlglot/schema.py
View file

@ -15,8 +15,6 @@ if t.TYPE_CHECKING:
ColumnMapping = t.Union[t.Dict, str, StructType, t.List] ColumnMapping = t.Union[t.Dict, str, StructType, t.List]
TABLE_ARGS = ("this", "db", "catalog")
class Schema(abc.ABC): class Schema(abc.ABC):
"""Abstract base class for database schemas""" """Abstract base class for database schemas"""
@ -147,7 +145,7 @@ class AbstractMappingSchema:
if not depth: # None if not depth: # None
self._supported_table_args = tuple() self._supported_table_args = tuple()
elif 1 <= depth <= 3: elif 1 <= depth <= 3:
self._supported_table_args = TABLE_ARGS[:depth] self._supported_table_args = exp.TABLE_PARTS[:depth]
else: else:
raise SchemaError(f"Invalid mapping shape. Depth: {depth}") raise SchemaError(f"Invalid mapping shape. Depth: {depth}")
@ -156,7 +154,7 @@ class AbstractMappingSchema:
def table_parts(self, table: exp.Table) -> t.List[str]: def table_parts(self, table: exp.Table) -> t.List[str]:
if isinstance(table.this, exp.ReadCSV): if isinstance(table.this, exp.ReadCSV):
return [table.this.name] return [table.this.name]
return [table.text(part) for part in TABLE_ARGS if table.text(part)] return [table.text(part) for part in exp.TABLE_PARTS if table.text(part)]
def find( def find(
self, table: exp.Table, trie: t.Optional[t.Dict] = None, raise_on_missing: bool = True self, table: exp.Table, trie: t.Optional[t.Dict] = None, raise_on_missing: bool = True
@ -365,13 +363,11 @@ class MappingSchema(AbstractMappingSchema, Schema):
f"Table {'.'.join(keys[:-1])} must match the schema's nesting level: {len(flattened_schema[0])}." f"Table {'.'.join(keys[:-1])} must match the schema's nesting level: {len(flattened_schema[0])}."
) )
normalized_keys = [ normalized_keys = [self._normalize_name(key, is_table=True) for key in keys]
self._normalize_name(key, dialect=self.dialect, is_table=True) for key in keys
]
for column_name, column_type in columns.items(): for column_name, column_type in columns.items():
nested_set( nested_set(
normalized_mapping, normalized_mapping,
normalized_keys + [self._normalize_name(column_name, dialect=self.dialect)], normalized_keys + [self._normalize_name(column_name)],
column_type, column_type,
) )
@ -383,20 +379,18 @@ class MappingSchema(AbstractMappingSchema, Schema):
dialect: DialectType = None, dialect: DialectType = None,
normalize: t.Optional[bool] = None, normalize: t.Optional[bool] = None,
) -> exp.Table: ) -> exp.Table:
normalized_table = exp.maybe_parse( dialect = dialect or self.dialect
table, into=exp.Table, dialect=dialect or self.dialect, copy=True normalize = self.normalize if normalize is None else normalize
)
for arg in TABLE_ARGS: normalized_table = exp.maybe_parse(table, into=exp.Table, dialect=dialect, copy=normalize)
if normalize:
for arg in exp.TABLE_PARTS:
value = normalized_table.args.get(arg) value = normalized_table.args.get(arg)
if isinstance(value, (str, exp.Identifier)): if isinstance(value, exp.Identifier):
normalized_table.set( normalized_table.set(
arg, arg,
exp.to_identifier( normalize_name(value, dialect=dialect, is_table=True, normalize=normalize),
self._normalize_name(
value, dialect=dialect, is_table=True, normalize=normalize
)
),
) )
return normalized_table return normalized_table
@ -413,7 +407,7 @@ class MappingSchema(AbstractMappingSchema, Schema):
dialect=dialect or self.dialect, dialect=dialect or self.dialect,
is_table=is_table, is_table=is_table,
normalize=self.normalize if normalize is None else normalize, normalize=self.normalize if normalize is None else normalize,
) ).name
def depth(self) -> int: def depth(self) -> int:
if not self.empty and not self._depth: if not self.empty and not self._depth:
@ -451,16 +445,16 @@ def normalize_name(
dialect: DialectType = None, dialect: DialectType = None,
is_table: bool = False, is_table: bool = False,
normalize: t.Optional[bool] = True, normalize: t.Optional[bool] = True,
) -> str: ) -> exp.Identifier:
if isinstance(identifier, str): if isinstance(identifier, str):
identifier = exp.parse_identifier(identifier, dialect=dialect) identifier = exp.parse_identifier(identifier, dialect=dialect)
if not normalize: if not normalize:
return identifier.name return identifier
# This can be useful for normalize_identifier # this is used for normalize_identifier, bigquery has special rules pertaining tables
identifier.meta["is_table"] = is_table identifier.meta["is_table"] = is_table
return Dialect.get_or_raise(dialect).normalize_identifier(identifier).name return Dialect.get_or_raise(dialect).normalize_identifier(identifier)
def ensure_schema(schema: Schema | t.Optional[t.Dict], **kwargs: t.Any) -> Schema: def ensure_schema(schema: Schema | t.Optional[t.Dict], **kwargs: t.Any) -> Schema:

4
sqlglot/time.py
View file

@ -42,6 +42,10 @@ def format_time(
end -= 1 end -= 1
chars = sym chars = sym
sym = None sym = None
else:
chars = chars[0]
end = start + 1
start += len(chars) start += len(chars)
chunks.append(chars) chunks.append(chars)
current = trie current = trie

66
sqlglot/tokens.py
View file

@ -7,6 +7,9 @@ from sqlglot.errors import TokenError
from sqlglot.helper import AutoName from sqlglot.helper import AutoName
from sqlglot.trie import TrieResult, in_trie, new_trie from sqlglot.trie import TrieResult, in_trie, new_trie
if t.TYPE_CHECKING:
from sqlglot.dialects.dialect import DialectType
class TokenType(AutoName): class TokenType(AutoName):
L_PAREN = auto() L_PAREN = auto()
@ -34,6 +37,7 @@ class TokenType(AutoName):
EQ = auto() EQ = auto()
NEQ = auto() NEQ = auto()
NULLSAFE_EQ = auto() NULLSAFE_EQ = auto()
COLON_EQ = auto()
AND = auto() AND = auto()
OR = auto() OR = auto()
AMP = auto() AMP = auto()
@ -56,6 +60,7 @@ class TokenType(AutoName):
SESSION_PARAMETER = auto() SESSION_PARAMETER = auto()
DAMP = auto() DAMP = auto()
XOR = auto() XOR = auto()
DSTAR = auto()
BLOCK_START = auto() BLOCK_START = auto()
BLOCK_END = auto() BLOCK_END = auto()
@ -274,6 +279,7 @@ class TokenType(AutoName):
OBJECT_IDENTIFIER = auto() OBJECT_IDENTIFIER = auto()
OFFSET = auto() OFFSET = auto()
ON = auto() ON = auto()
OPERATOR = auto()
ORDER_BY = auto() ORDER_BY = auto()
ORDERED = auto() ORDERED = auto()
ORDINALITY = auto() ORDINALITY = auto()
@ -295,6 +301,7 @@ class TokenType(AutoName):
QUOTE = auto() QUOTE = auto()
RANGE = auto() RANGE = auto()
RECURSIVE = auto() RECURSIVE = auto()
REFRESH = auto()
REPLACE = auto() REPLACE = auto()
RETURNING = auto() RETURNING = auto()
REFERENCES = auto() REFERENCES = auto()
@ -371,7 +378,7 @@ class Token:
col: int = 1, col: int = 1,
start: int = 0, start: int = 0,
end: int = 0, end: int = 0,
comments: t.List[str] = [], comments: t.Optional[t.List[str]] = None,
) -> None: ) -> None:
"""Token initializer. """Token initializer.
@ -390,7 +397,7 @@ class Token:
self.col = col self.col = col
self.start = start self.start = start
self.end = end self.end = end
self.comments = comments self.comments = [] if comments is None else comments
def __repr__(self) -> str: def __repr__(self) -> str:
attributes = ", ".join(f"{k}: {getattr(self, k)}" for k in self.__slots__) attributes = ", ".join(f"{k}: {getattr(self, k)}" for k in self.__slots__)
@ -497,11 +504,8 @@ class Tokenizer(metaclass=_Tokenizer):
QUOTES: t.List[t.Tuple[str, str] | str] = ["'"] QUOTES: t.List[t.Tuple[str, str] | str] = ["'"]
STRING_ESCAPES = ["'"] STRING_ESCAPES = ["'"]
VAR_SINGLE_TOKENS: t.Set[str] = set() VAR_SINGLE_TOKENS: t.Set[str] = set()
ESCAPE_SEQUENCES: t.Dict[str, str] = {}
# Autofilled # Autofilled
IDENTIFIERS_CAN_START_WITH_DIGIT: bool = False
_COMMENTS: t.Dict[str, str] = {} _COMMENTS: t.Dict[str, str] = {}
_FORMAT_STRINGS: t.Dict[str, t.Tuple[str, TokenType]] = {} _FORMAT_STRINGS: t.Dict[str, t.Tuple[str, TokenType]] = {}
_IDENTIFIERS: t.Dict[str, str] = {} _IDENTIFIERS: t.Dict[str, str] = {}
@ -523,6 +527,7 @@ class Tokenizer(metaclass=_Tokenizer):
"<=": TokenType.LTE, "<=": TokenType.LTE,
"<>": TokenType.NEQ, "<>": TokenType.NEQ,
"!=": TokenType.NEQ, "!=": TokenType.NEQ,
":=": TokenType.COLON_EQ,
"<=>": TokenType.NULLSAFE_EQ, "<=>": TokenType.NULLSAFE_EQ,
"->": TokenType.ARROW, "->": TokenType.ARROW,
"->>": TokenType.DARROW, "->>": TokenType.DARROW,
@ -689,17 +694,22 @@ class Tokenizer(metaclass=_Tokenizer):
"BOOLEAN": TokenType.BOOLEAN, "BOOLEAN": TokenType.BOOLEAN,
"BYTE": TokenType.TINYINT, "BYTE": TokenType.TINYINT,
"MEDIUMINT": TokenType.MEDIUMINT, "MEDIUMINT": TokenType.MEDIUMINT,
"INT1": TokenType.TINYINT,
"TINYINT": TokenType.TINYINT, "TINYINT": TokenType.TINYINT,
"INT16": TokenType.SMALLINT,
"SHORT": TokenType.SMALLINT, "SHORT": TokenType.SMALLINT,
"SMALLINT": TokenType.SMALLINT, "SMALLINT": TokenType.SMALLINT,
"INT128": TokenType.INT128, "INT128": TokenType.INT128,
"HUGEINT": TokenType.INT128,
"INT2": TokenType.SMALLINT, "INT2": TokenType.SMALLINT,
"INTEGER": TokenType.INT, "INTEGER": TokenType.INT,
"INT": TokenType.INT, "INT": TokenType.INT,
"INT4": TokenType.INT, "INT4": TokenType.INT,
"INT32": TokenType.INT,
"INT64": TokenType.BIGINT,
"LONG": TokenType.BIGINT, "LONG": TokenType.BIGINT,
"BIGINT": TokenType.BIGINT, "BIGINT": TokenType.BIGINT,
"INT8": TokenType.BIGINT, "INT8": TokenType.TINYINT,
"DEC": TokenType.DECIMAL, "DEC": TokenType.DECIMAL,
"DECIMAL": TokenType.DECIMAL, "DECIMAL": TokenType.DECIMAL,
"BIGDECIMAL": TokenType.BIGDECIMAL, "BIGDECIMAL": TokenType.BIGDECIMAL,
@ -781,7 +791,6 @@ class Tokenizer(metaclass=_Tokenizer):
"\t": TokenType.SPACE, "\t": TokenType.SPACE,
"\n": TokenType.BREAK, "\n": TokenType.BREAK,
"\r": TokenType.BREAK, "\r": TokenType.BREAK,
"\r\n": TokenType.BREAK,
} }
COMMANDS = { COMMANDS = {
@ -803,6 +812,7 @@ class Tokenizer(metaclass=_Tokenizer):
"sql", "sql",
"size", "size",
"tokens", "tokens",
"dialect",
"_start", "_start",
"_current", "_current",
"_line", "_line",
@ -814,7 +824,10 @@ class Tokenizer(metaclass=_Tokenizer):
"_prev_token_line", "_prev_token_line",
) )
def __init__(self) -> None: def __init__(self, dialect: DialectType = None) -> None:
from sqlglot.dialects import Dialect
self.dialect = Dialect.get_or_raise(dialect)
self.reset() self.reset()
def reset(self) -> None: def reset(self) -> None:
@ -850,13 +863,26 @@ class Tokenizer(metaclass=_Tokenizer):
def _scan(self, until: t.Optional[t.Callable] = None) -> None: def _scan(self, until: t.Optional[t.Callable] = None) -> None:
while self.size and not self._end: while self.size and not self._end:
self._start = self._current current = self._current
self._advance()
# skip spaces inline rather than iteratively call advance()
# for performance reasons
while current < self.size:
char = self.sql[current]
if char.isspace() and (char == " " or char == "\t"):
current += 1
else:
break
n = current - self._current
self._start = current
self._advance(n if n > 1 else 1)
if self._char is None: if self._char is None:
break break
if self._char not in self.WHITE_SPACE: if not self._char.isspace():
if self._char.isdigit(): if self._char.isdigit():
self._scan_number() self._scan_number()
elif self._char in self._IDENTIFIERS: elif self._char in self._IDENTIFIERS:
@ -881,6 +907,10 @@ class Tokenizer(metaclass=_Tokenizer):
def _advance(self, i: int = 1, alnum: bool = False) -> None: def _advance(self, i: int = 1, alnum: bool = False) -> None:
if self.WHITE_SPACE.get(self._char) is TokenType.BREAK: if self.WHITE_SPACE.get(self._char) is TokenType.BREAK:
# Ensures we don't count an extra line if we get a \r\n line break sequence
if self._char == "\r" and self._peek == "\n":
i = 2
self._col = 1 self._col = 1
self._line += 1 self._line += 1
else: else:
@ -982,7 +1012,7 @@ class Tokenizer(metaclass=_Tokenizer):
if end < self.size: if end < self.size:
char = self.sql[end] char = self.sql[end]
single_token = single_token or char in self.SINGLE_TOKENS single_token = single_token or char in self.SINGLE_TOKENS
is_space = char in self.WHITE_SPACE is_space = char.isspace()
if not is_space or not prev_space: if not is_space or not prev_space:
if is_space: if is_space:
@ -994,7 +1024,7 @@ class Tokenizer(metaclass=_Tokenizer):
skip = True skip = True
else: else:
char = "" char = ""
chars = " " break
if word: if word:
if self._scan_string(word): if self._scan_string(word):
@ -1086,7 +1116,7 @@ class Tokenizer(metaclass=_Tokenizer):
self._add(TokenType.NUMBER, number_text) self._add(TokenType.NUMBER, number_text)
self._add(TokenType.DCOLON, "::") self._add(TokenType.DCOLON, "::")
return self._add(token_type, literal) return self._add(token_type, literal)
elif self.IDENTIFIERS_CAN_START_WITH_DIGIT: elif self.dialect.IDENTIFIERS_CAN_START_WITH_DIGIT:
return self._add(TokenType.VAR) return self._add(TokenType.VAR)
self._advance(-len(literal)) self._advance(-len(literal))
@ -1208,8 +1238,12 @@ class Tokenizer(metaclass=_Tokenizer):
if self._end: if self._end:
raise TokenError(f"Missing {delimiter} from {self._line}:{self._start}") raise TokenError(f"Missing {delimiter} from {self._line}:{self._start}")
if self.ESCAPE_SEQUENCES and self._peek and self._char in self.STRING_ESCAPES: if (
escaped_sequence = self.ESCAPE_SEQUENCES.get(self._char + self._peek) self.dialect.ESCAPE_SEQUENCES
and self._peek
and self._char in self.STRING_ESCAPES
):
escaped_sequence = self.dialect.ESCAPE_SEQUENCES.get(self._char + self._peek)
if escaped_sequence: if escaped_sequence:
self._advance(2) self._advance(2)
text += escaped_sequence text += escaped_sequence

62
sqlglot/transforms.py
View file

@ -141,7 +141,7 @@ def remove_precision_parameterized_types(expression: exp.Expression) -> exp.Expr
def unnest_to_explode(expression: exp.Expression) -> exp.Expression: def unnest_to_explode(expression: exp.Expression) -> exp.Expression:
"""Convert cross join unnest into lateral view explode (used in presto -> hive).""" """Convert cross join unnest into lateral view explode."""
if isinstance(expression, exp.Select): if isinstance(expression, exp.Select):
for join in expression.args.get("joins") or []: for join in expression.args.get("joins") or []:
unnest = join.this unnest = join.this
@ -166,7 +166,7 @@ def unnest_to_explode(expression: exp.Expression) -> exp.Expression:
def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp.Expression]: def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp.Expression]:
"""Convert explode/posexplode into unnest (used in hive -> presto).""" """Convert explode/posexplode into unnest."""
def _explode_to_unnest(expression: exp.Expression) -> exp.Expression: def _explode_to_unnest(expression: exp.Expression) -> exp.Expression:
if isinstance(expression, exp.Select): if isinstance(expression, exp.Select):
@ -199,11 +199,11 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
explode_alias = "" explode_alias = ""
if isinstance(select, exp.Alias): if isinstance(select, exp.Alias):
explode_alias = select.alias explode_alias = select.args["alias"]
alias = select alias = select
elif isinstance(select, exp.Aliases): elif isinstance(select, exp.Aliases):
pos_alias = select.aliases[0].name pos_alias = select.aliases[0]
explode_alias = select.aliases[1].name explode_alias = select.aliases[1]
alias = select.replace(exp.alias_(select.this, "", copy=False)) alias = select.replace(exp.alias_(select.this, "", copy=False))
else: else:
alias = select.replace(exp.alias_(select, "")) alias = select.replace(exp.alias_(select, ""))
@ -230,9 +230,12 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
alias.set("alias", exp.to_identifier(explode_alias)) alias.set("alias", exp.to_identifier(explode_alias))
series_table_alias = series.args["alias"].this
column = exp.If( column = exp.If(
this=exp.column(series_alias).eq(exp.column(pos_alias)), this=exp.column(series_alias, table=series_table_alias).eq(
true=exp.column(explode_alias), exp.column(pos_alias, table=unnest_source_alias)
),
true=exp.column(explode_alias, table=unnest_source_alias),
) )
explode.replace(column) explode.replace(column)
@ -242,8 +245,10 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
expressions.insert( expressions.insert(
expressions.index(alias) + 1, expressions.index(alias) + 1,
exp.If( exp.If(
this=exp.column(series_alias).eq(exp.column(pos_alias)), this=exp.column(series_alias, table=series_table_alias).eq(
true=exp.column(pos_alias), exp.column(pos_alias, table=unnest_source_alias)
),
true=exp.column(pos_alias, table=unnest_source_alias),
).as_(pos_alias), ).as_(pos_alias),
) )
expression.set("expressions", expressions) expression.set("expressions", expressions)
@ -276,10 +281,12 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
size = size - 1 size = size - 1
expression.where( expression.where(
exp.column(series_alias) exp.column(series_alias, table=series_table_alias)
.eq(exp.column(pos_alias)) .eq(exp.column(pos_alias, table=unnest_source_alias))
.or_( .or_(
(exp.column(series_alias) > size).and_(exp.column(pos_alias).eq(size)) (exp.column(series_alias, table=series_table_alias) > size).and_(
exp.column(pos_alias, table=unnest_source_alias).eq(size)
)
), ),
copy=False, copy=False,
) )
@ -386,14 +393,16 @@ def eliminate_full_outer_join(expression: exp.Expression) -> exp.Expression:
full_outer_joins = [ full_outer_joins = [
(index, join) (index, join)
for index, join in enumerate(expression.args.get("joins") or []) for index, join in enumerate(expression.args.get("joins") or [])
if join.side == "FULL" and join.kind == "OUTER" if join.side == "FULL"
] ]
if len(full_outer_joins) == 1: if len(full_outer_joins) == 1:
expression_copy = expression.copy() expression_copy = expression.copy()
expression.set("limit", None)
index, full_outer_join = full_outer_joins[0] index, full_outer_join = full_outer_joins[0]
full_outer_join.set("side", "left") full_outer_join.set("side", "left")
expression_copy.args["joins"][index].set("side", "right") expression_copy.args["joins"][index].set("side", "right")
expression_copy.args.pop("with", None) # remove CTEs from RIGHT side
return exp.union(expression, expression_copy, copy=False) return exp.union(expression, expression_copy, copy=False)
@ -430,6 +439,33 @@ def move_ctes_to_top_level(expression: exp.Expression) -> exp.Expression:
return expression return expression
def ensure_bools(expression: exp.Expression) -> exp.Expression:
"""Converts numeric values used in conditions into explicit boolean expressions."""
from sqlglot.optimizer.canonicalize import ensure_bools
def _ensure_bool(node: exp.Expression) -> None:
if (
node.is_number
or node.is_type(exp.DataType.Type.UNKNOWN, *exp.DataType.NUMERIC_TYPES)
or (isinstance(node, exp.Column) and not node.type)
):
node.replace(node.neq(0))
for node, *_ in expression.walk():
ensure_bools(node, _ensure_bool)
return expression
def unqualify_columns(expression: exp.Expression) -> exp.Expression:
for column in expression.find_all(exp.Column):
# We only wanna pop off the table, db, catalog args
for part in column.parts[:-1]:
part.pop()
return expression
def preprocess( def preprocess(
transforms: t.List[t.Callable[[exp.Expression], exp.Expression]], transforms: t.List[t.Callable[[exp.Expression], exp.Expression]],
) -> t.Callable[[Generator, exp.Expression], str]: ) -> t.Callable[[Generator, exp.Expression], str]:

2
tests/dataframe/unit/test_column.py
View file

@ -146,7 +146,7 @@ class TestDataframeColumn(unittest.TestCase):
self.assertEqual("cola BETWEEN 1 AND 3", F.col("cola").between(1, 3).sql()) self.assertEqual("cola BETWEEN 1 AND 3", F.col("cola").between(1, 3).sql())
self.assertEqual("cola BETWEEN 10.1 AND 12.1", F.col("cola").between(10.1, 12.1).sql()) self.assertEqual("cola BETWEEN 10.1 AND 12.1", F.col("cola").between(10.1, 12.1).sql())
self.assertEqual( self.assertEqual(
"cola BETWEEN TO_DATE('2022-01-01') AND TO_DATE('2022-03-01')", "cola BETWEEN CAST('2022-01-01' AS DATE) AND CAST('2022-03-01' AS DATE)",
F.col("cola").between(datetime.date(2022, 1, 1), datetime.date(2022, 3, 1)).sql(), F.col("cola").between(datetime.date(2022, 1, 1), datetime.date(2022, 3, 1)).sql(),
) )
self.assertEqual( self.assertEqual(

4
tests/dataframe/unit/test_functions.py
View file

@ -27,7 +27,7 @@ class TestFunctions(unittest.TestCase):
test_null = SF.lit(None) test_null = SF.lit(None)
self.assertEqual("NULL", test_null.sql()) self.assertEqual("NULL", test_null.sql())
test_date = SF.lit(datetime.date(2022, 1, 1)) test_date = SF.lit(datetime.date(2022, 1, 1))
self.assertEqual("TO_DATE('2022-01-01')", test_date.sql()) self.assertEqual("CAST('2022-01-01' AS DATE)", test_date.sql())
test_datetime = SF.lit(datetime.datetime(2022, 1, 1, 1, 1, 1)) test_datetime = SF.lit(datetime.datetime(2022, 1, 1, 1, 1, 1))
self.assertEqual("CAST('2022-01-01T01:01:01+00:00' AS TIMESTAMP)", test_datetime.sql()) self.assertEqual("CAST('2022-01-01T01:01:01+00:00' AS TIMESTAMP)", test_datetime.sql())
test_dict = SF.lit({"cola": 1, "colb": "test"}) test_dict = SF.lit({"cola": 1, "colb": "test"})
@ -49,7 +49,7 @@ class TestFunctions(unittest.TestCase):
test_array = SF.col([1, 2, "3"]) test_array = SF.col([1, 2, "3"])
self.assertEqual("ARRAY(1, 2, '3')", test_array.sql()) self.assertEqual("ARRAY(1, 2, '3')", test_array.sql())
test_date = SF.col(datetime.date(2022, 1, 1)) test_date = SF.col(datetime.date(2022, 1, 1))
self.assertEqual("TO_DATE('2022-01-01')", test_date.sql()) self.assertEqual("CAST('2022-01-01' AS DATE)", test_date.sql())
test_datetime = SF.col(datetime.datetime(2022, 1, 1, 1, 1, 1)) test_datetime = SF.col(datetime.datetime(2022, 1, 1, 1, 1, 1))
self.assertEqual("CAST('2022-01-01T01:01:01+00:00' AS TIMESTAMP)", test_datetime.sql()) self.assertEqual("CAST('2022-01-01T01:01:01+00:00' AS TIMESTAMP)", test_datetime.sql())
test_dict = SF.col({"cola": 1, "colb": "test"}) test_dict = SF.col({"cola": 1, "colb": "test"})

59
tests/dialects/test_bigquery.py
View file

@ -1,6 +1,14 @@
from unittest import mock from unittest import mock
from sqlglot import ErrorLevel, ParseError, TokenError, UnsupportedError, transpile from sqlglot import (
ErrorLevel,
ParseError,
TokenError,
UnsupportedError,
parse,
transpile,
)
from sqlglot.helper import logger as helper_logger
from tests.dialects.test_dialect import Validator from tests.dialects.test_dialect import Validator
@ -9,6 +17,28 @@ class TestBigQuery(Validator):
maxDiff = None maxDiff = None
def test_bigquery(self): def test_bigquery(self):
with self.assertLogs(helper_logger) as cm:
self.validate_all(
"SELECT a[1], b[OFFSET(1)], c[ORDINAL(1)], d[SAFE_OFFSET(1)], e[SAFE_ORDINAL(1)]",
write={
"duckdb": "SELECT a[2], b[2], c[1], d[2], e[1]",
"bigquery": "SELECT a[1], b[OFFSET(1)], c[ORDINAL(1)], d[SAFE_OFFSET(1)], e[SAFE_ORDINAL(1)]",
"presto": "SELECT a[2], b[2], c[1], ELEMENT_AT(d, 2), ELEMENT_AT(e, 1)",
},
)
self.validate_all(
"a[0]",
read={
"duckdb": "a[1]",
"presto": "a[1]",
},
)
self.validate_identity(
"select array_contains([1, 2, 3], 1)",
"SELECT EXISTS(SELECT 1 FROM UNNEST([1, 2, 3]) AS _col WHERE _col = 1)",
)
self.validate_identity("CREATE SCHEMA x DEFAULT COLLATE 'en'") self.validate_identity("CREATE SCHEMA x DEFAULT COLLATE 'en'")
self.validate_identity("CREATE TABLE x (y INT64) DEFAULT COLLATE 'en'") self.validate_identity("CREATE TABLE x (y INT64) DEFAULT COLLATE 'en'")
self.validate_identity("PARSE_JSON('{}', wide_number_mode => 'exact')") self.validate_identity("PARSE_JSON('{}', wide_number_mode => 'exact')")
@ -37,6 +67,15 @@ class TestBigQuery(Validator):
with self.assertRaises(ParseError): with self.assertRaises(ParseError):
transpile("DATE_ADD(x, day)", read="bigquery") transpile("DATE_ADD(x, day)", read="bigquery")
for_in_stmts = parse(
"FOR record IN (SELECT word FROM shakespeare) DO SELECT record.word; END FOR;",
read="bigquery",
)
self.assertEqual(
[s.sql(dialect="bigquery") for s in for_in_stmts],
["FOR record IN (SELECT word FROM shakespeare) DO SELECT record.word", "END FOR"],
)
self.validate_identity("SELECT test.Unknown FROM test") self.validate_identity("SELECT test.Unknown FROM test")
self.validate_identity(r"SELECT '\n\r\a\v\f\t'") self.validate_identity(r"SELECT '\n\r\a\v\f\t'")
self.validate_identity("SELECT * FROM tbl FOR SYSTEM_TIME AS OF z") self.validate_identity("SELECT * FROM tbl FOR SYSTEM_TIME AS OF z")
@ -89,6 +128,11 @@ class TestBigQuery(Validator):
self.validate_identity("ROLLBACK TRANSACTION") self.validate_identity("ROLLBACK TRANSACTION")
self.validate_identity("CAST(x AS BIGNUMERIC)") self.validate_identity("CAST(x AS BIGNUMERIC)")
self.validate_identity("SELECT y + 1 FROM x GROUP BY y + 1 ORDER BY 1") self.validate_identity("SELECT y + 1 FROM x GROUP BY y + 1 ORDER BY 1")
self.validate_identity("SELECT TIMESTAMP_SECONDS(2) AS t")
self.validate_identity("SELECT TIMESTAMP_MILLIS(2) AS t")
self.validate_identity(
"FOR record IN (SELECT word, word_count FROM bigquery-public-data.samples.shakespeare LIMIT 5) DO SELECT record.word, record.word_count"
)
self.validate_identity( self.validate_identity(
"DATE(CAST('2016-12-25 05:30:00+07' AS DATETIME), 'America/Los_Angeles')" "DATE(CAST('2016-12-25 05:30:00+07' AS DATETIME), 'America/Los_Angeles')"
) )
@ -142,6 +186,19 @@ class TestBigQuery(Validator):
self.validate_all('x <> """"""', write={"bigquery": "x <> ''"}) self.validate_all('x <> """"""', write={"bigquery": "x <> ''"})
self.validate_all("x <> ''''''", write={"bigquery": "x <> ''"}) self.validate_all("x <> ''''''", write={"bigquery": "x <> ''"})
self.validate_all("CAST(x AS DATETIME)", read={"": "x::timestamp"}) self.validate_all("CAST(x AS DATETIME)", read={"": "x::timestamp"})
self.validate_all(
"SELECT TIMESTAMP_MICROS(x)",
read={
"duckdb": "SELECT MAKE_TIMESTAMP(x)",
"spark": "SELECT TIMESTAMP_MICROS(x)",
},
write={
"bigquery": "SELECT TIMESTAMP_MICROS(x)",
"duckdb": "SELECT MAKE_TIMESTAMP(x)",
"snowflake": "SELECT TO_TIMESTAMP(x / 1000, 3)",
"spark": "SELECT TIMESTAMP_MICROS(x)",
},
)
self.validate_all( self.validate_all(
"SELECT * FROM t WHERE EXISTS(SELECT * FROM unnest(nums) AS x WHERE x > 1)", "SELECT * FROM t WHERE EXISTS(SELECT * FROM unnest(nums) AS x WHERE x > 1)",
write={ write={

68
tests/dialects/test_clickhouse.py
View file

@ -6,22 +6,6 @@ class TestClickhouse(Validator):
dialect = "clickhouse" dialect = "clickhouse"
def test_clickhouse(self): def test_clickhouse(self):
self.validate_identity("x <> y")
self.validate_all(
"has([1], x)",
read={
"postgres": "x = any(array[1])",
},
)
self.validate_all(
"NOT has([1], x)",
read={
"postgres": "any(array[1]) <> x",
},
)
self.validate_identity("x = y")
string_types = [ string_types = [
"BLOB", "BLOB",
"LONGBLOB", "LONGBLOB",
@ -40,6 +24,8 @@ class TestClickhouse(Validator):
self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)") self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)")
self.assertIsNone(expr._meta) self.assertIsNone(expr._meta)
self.validate_identity("x = y")
self.validate_identity("x <> y")
self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 0.01)") self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 0.01)")
self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 1 / 10 OFFSET 1 / 2)") self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 1 / 10 OFFSET 1 / 2)")
self.validate_identity("SELECT sum(foo * bar) FROM bla SAMPLE 10000000") self.validate_identity("SELECT sum(foo * bar) FROM bla SAMPLE 10000000")
@ -81,7 +67,17 @@ class TestClickhouse(Validator):
self.validate_identity("position(haystack, needle, position)") self.validate_identity("position(haystack, needle, position)")
self.validate_identity("CAST(x AS DATETIME)") self.validate_identity("CAST(x AS DATETIME)")
self.validate_identity("CAST(x as MEDIUMINT)", "CAST(x AS Int32)") self.validate_identity("CAST(x as MEDIUMINT)", "CAST(x AS Int32)")
self.validate_identity("SELECT arrayJoin([1, 2, 3] AS src) AS dst, 'Hello', src")
self.validate_identity(
"SELECT SUM(1) AS impressions, arrayJoin(cities) AS city, arrayJoin(browsers) AS browser FROM (SELECT ['Istanbul', 'Berlin', 'Bobruisk'] AS cities, ['Firefox', 'Chrome', 'Chrome'] AS browsers) GROUP BY 2, 3"
)
self.validate_identity(
"SELECT sum(1) AS impressions, (arrayJoin(arrayZip(cities, browsers)) AS t).1 AS city, t.2 AS browser FROM (SELECT ['Istanbul', 'Berlin', 'Bobruisk'] AS cities, ['Firefox', 'Chrome', 'Chrome'] AS browsers) GROUP BY 2, 3"
)
self.validate_identity(
"SELECT SUM(1) AS impressions FROM (SELECT ['Istanbul', 'Berlin', 'Bobruisk'] AS cities) WHERE arrayJoin(cities) IN ['Istanbul', 'Berlin']",
"SELECT SUM(1) AS impressions FROM (SELECT ['Istanbul', 'Berlin', 'Bobruisk'] AS cities) WHERE arrayJoin(cities) IN ('Istanbul', 'Berlin')",
)
self.validate_identity( self.validate_identity(
'SELECT CAST(tuple(1 AS "a", 2 AS "b", 3.0 AS "c").2 AS Nullable(String))' 'SELECT CAST(tuple(1 AS "a", 2 AS "b", 3.0 AS "c").2 AS Nullable(String))'
) )
@ -101,6 +97,25 @@ class TestClickhouse(Validator):
"CREATE MATERIALIZED VIEW test_view (id UInt8) TO db.table1 AS SELECT * FROM test_data" "CREATE MATERIALIZED VIEW test_view (id UInt8) TO db.table1 AS SELECT * FROM test_data"
) )
self.validate_all(
"SELECT arrayJoin([1,2,3])",
write={
"clickhouse": "SELECT arrayJoin([1, 2, 3])",
"postgres": "SELECT UNNEST(ARRAY[1, 2, 3])",
},
)
self.validate_all(
"has([1], x)",
read={
"postgres": "x = any(array[1])",
},
)
self.validate_all(
"NOT has([1], x)",
read={
"postgres": "any(array[1]) <> x",
},
)
self.validate_all( self.validate_all(
"SELECT CAST('2020-01-01' AS TIMESTAMP) + INTERVAL '500' microsecond", "SELECT CAST('2020-01-01' AS TIMESTAMP) + INTERVAL '500' microsecond",
read={ read={
@ -197,12 +212,15 @@ class TestClickhouse(Validator):
}, },
) )
self.validate_all( self.validate_all(
"CONCAT(CASE WHEN COALESCE(CAST(a AS String), '') IS NULL THEN COALESCE(CAST(a AS String), '') ELSE CAST(COALESCE(CAST(a AS String), '') AS String) END, CASE WHEN COALESCE(CAST(b AS String), '') IS NULL THEN COALESCE(CAST(b AS String), '') ELSE CAST(COALESCE(CAST(b AS String), '') AS String) END)", "CONCAT(a, b)",
read={"postgres": "CONCAT(a, b)"}, read={
) "clickhouse": "CONCAT(a, b)",
self.validate_all( "mysql": "CONCAT(a, b)",
"CONCAT(CASE WHEN a IS NULL THEN a ELSE CAST(a AS String) END, CASE WHEN b IS NULL THEN b ELSE CAST(b AS String) END)", },
read={"mysql": "CONCAT(a, b)"}, write={
"mysql": "CONCAT(a, b)",
"postgres": "CONCAT(a, b)",
},
) )
self.validate_all( self.validate_all(
r"'Enum8(\'Sunday\' = 0)'", write={"clickhouse": "'Enum8(''Sunday'' = 0)'"} r"'Enum8(\'Sunday\' = 0)'", write={"clickhouse": "'Enum8(''Sunday'' = 0)'"}
@ -320,6 +338,10 @@ class TestClickhouse(Validator):
self.validate_identity("WITH (SELECT foo) AS bar SELECT bar + 5") self.validate_identity("WITH (SELECT foo) AS bar SELECT bar + 5")
self.validate_identity("WITH test1 AS (SELECT i + 1, j + 1 FROM test1) SELECT * FROM test1") self.validate_identity("WITH test1 AS (SELECT i + 1, j + 1 FROM test1) SELECT * FROM test1")
query = parse_one("""WITH (SELECT 1) AS y SELECT * FROM y""", read="clickhouse")
self.assertIsInstance(query.args["with"].expressions[0].this, exp.Subquery)
self.assertEqual(query.args["with"].expressions[0].alias, "y")
def test_ternary(self): def test_ternary(self):
self.validate_all("x ? 1 : 2", write={"clickhouse": "CASE WHEN x THEN 1 ELSE 2 END"}) self.validate_all("x ? 1 : 2", write={"clickhouse": "CASE WHEN x THEN 1 ELSE 2 END"})
self.validate_all( self.validate_all(

2
tests/dialects/test_databricks.py
View file

@ -131,7 +131,7 @@ class TestDatabricks(Validator):
"SELECT DATEDIFF(week, 'start', 'end')", "SELECT DATEDIFF(week, 'start', 'end')",
write={ write={
"databricks": "SELECT DATEDIFF(week, 'start', 'end')", "databricks": "SELECT DATEDIFF(week, 'start', 'end')",
"postgres": "SELECT CAST(EXTRACT(year FROM AGE(CAST('end' AS TIMESTAMP), CAST('start' AS TIMESTAMP))) * 48 + EXTRACT(month FROM AGE(CAST('end' AS TIMESTAMP), CAST('start' AS TIMESTAMP))) * 4 + EXTRACT(day FROM AGE(CAST('end' AS TIMESTAMP), CAST('start' AS TIMESTAMP))) / 7 AS BIGINT)", "postgres": "SELECT CAST(EXTRACT(days FROM (CAST('end' AS TIMESTAMP) - CAST('start' AS TIMESTAMP))) / 7 AS BIGINT)",
}, },
) )
self.validate_all( self.validate_all(

219
tests/dialects/test_dialect.py
View file

@ -7,9 +7,10 @@ from sqlglot import (
ParseError, ParseError,
TokenError, TokenError,
UnsupportedError, UnsupportedError,
exp,
parse_one, parse_one,
) )
from sqlglot.dialects import Hive from sqlglot.dialects import BigQuery, Hive, Snowflake
class Validator(unittest.TestCase): class Validator(unittest.TestCase):
@ -78,9 +79,56 @@ class TestDialect(Validator):
self.assertIsNotNone(Dialect[dialect.value]) self.assertIsNotNone(Dialect[dialect.value])
def test_get_or_raise(self): def test_get_or_raise(self):
self.assertEqual(Dialect.get_or_raise(Hive), Hive) self.assertIsInstance(Dialect.get_or_raise(Hive), Hive)
self.assertEqual(Dialect.get_or_raise(Hive()), Hive) self.assertIsInstance(Dialect.get_or_raise(Hive()), Hive)
self.assertEqual(Dialect.get_or_raise("hive"), Hive) self.assertIsInstance(Dialect.get_or_raise("hive"), Hive)
with self.assertRaises(ValueError):
Dialect.get_or_raise(1)
default_mysql = Dialect.get_or_raise("mysql")
self.assertEqual(default_mysql.normalization_strategy, "CASE_SENSITIVE")
lowercase_mysql = Dialect.get_or_raise("mysql,normalization_strategy=lowercase")
self.assertEqual(lowercase_mysql.normalization_strategy, "LOWERCASE")
lowercase_mysql = Dialect.get_or_raise("mysql, normalization_strategy = lowercase")
self.assertEqual(lowercase_mysql.normalization_strategy.value, "LOWERCASE")
with self.assertRaises(ValueError) as cm:
Dialect.get_or_raise("mysql, normalization_strategy")
self.assertEqual(
str(cm.exception),
"Invalid dialect format: 'mysql, normalization_strategy'. "
"Please use the correct format: 'dialect [, k1 = v2 [, ...]]'.",
)
def test_compare_dialects(self):
bigquery_class = Dialect["bigquery"]
bigquery_object = BigQuery()
bigquery_string = "bigquery"
snowflake_class = Dialect["snowflake"]
snowflake_object = Snowflake()
snowflake_string = "snowflake"
self.assertEqual(snowflake_class, snowflake_class)
self.assertEqual(snowflake_class, snowflake_object)
self.assertEqual(snowflake_class, snowflake_string)
self.assertEqual(snowflake_object, snowflake_object)
self.assertEqual(snowflake_object, snowflake_string)
self.assertNotEqual(snowflake_class, bigquery_class)
self.assertNotEqual(snowflake_class, bigquery_object)
self.assertNotEqual(snowflake_class, bigquery_string)
self.assertNotEqual(snowflake_object, bigquery_object)
self.assertNotEqual(snowflake_object, bigquery_string)
self.assertTrue(snowflake_class in {"snowflake", "bigquery"})
self.assertTrue(snowflake_object in {"snowflake", "bigquery"})
self.assertFalse(snowflake_class in {"bigquery", "redshift"})
self.assertFalse(snowflake_object in {"bigquery", "redshift"})
def test_cast(self): def test_cast(self):
self.validate_all( self.validate_all(
@ -561,6 +609,7 @@ class TestDialect(Validator):
self.validate_all( self.validate_all(
"TIME_TO_STR(x, '%Y-%m-%d')", "TIME_TO_STR(x, '%Y-%m-%d')",
write={ write={
"bigquery": "FORMAT_DATE('%Y-%m-%d', x)",
"drill": "TO_CHAR(x, 'yyyy-MM-dd')", "drill": "TO_CHAR(x, 'yyyy-MM-dd')",
"duckdb": "STRFTIME(x, '%Y-%m-%d')", "duckdb": "STRFTIME(x, '%Y-%m-%d')",
"hive": "DATE_FORMAT(x, 'yyyy-MM-dd')", "hive": "DATE_FORMAT(x, 'yyyy-MM-dd')",
@ -866,9 +915,9 @@ class TestDialect(Validator):
write={ write={
"drill": "CAST(x AS DATE)", "drill": "CAST(x AS DATE)",
"duckdb": "CAST(x AS DATE)", "duckdb": "CAST(x AS DATE)",
"hive": "TO_DATE(x)", "hive": "CAST(x AS DATE)",
"presto": "CAST(DATE_PARSE(x, '%Y-%m-%d') AS DATE)", "presto": "CAST(x AS DATE)",
"spark": "TO_DATE(x)", "spark": "CAST(x AS DATE)",
"sqlite": "x", "sqlite": "x",
}, },
) )
@ -893,7 +942,7 @@ class TestDialect(Validator):
self.validate_all( self.validate_all(
"TS_OR_DS_ADD(CURRENT_DATE, 1, 'DAY')", "TS_OR_DS_ADD(CURRENT_DATE, 1, 'DAY')",
write={ write={
"presto": "DATE_ADD('DAY', 1, CURRENT_DATE)", "presto": "DATE_ADD('DAY', 1, CAST(CAST(CURRENT_DATE AS TIMESTAMP) AS DATE))",
"hive": "DATE_ADD(CURRENT_DATE, 1)", "hive": "DATE_ADD(CURRENT_DATE, 1)",
}, },
) )
@ -1268,13 +1317,6 @@ class TestDialect(Validator):
"doris": "LOWER(x) LIKE '%y'", "doris": "LOWER(x) LIKE '%y'",
}, },
) )
self.validate_all(
"SELECT * FROM a ORDER BY col_a NULLS LAST",
write={
"mysql": UnsupportedError,
"starrocks": UnsupportedError,
},
)
self.validate_all( self.validate_all(
"POSITION(needle in haystack)", "POSITION(needle in haystack)",
write={ write={
@ -1315,35 +1357,37 @@ class TestDialect(Validator):
self.validate_all( self.validate_all(
"CONCAT_WS('-', 'a', 'b')", "CONCAT_WS('-', 'a', 'b')",
write={ write={
"clickhouse": "CONCAT_WS('-', 'a', 'b')",
"duckdb": "CONCAT_WS('-', 'a', 'b')", "duckdb": "CONCAT_WS('-', 'a', 'b')",
"presto": "CONCAT_WS('-', 'a', 'b')", "presto": "CONCAT_WS('-', CAST('a' AS VARCHAR), CAST('b' AS VARCHAR))",
"hive": "CONCAT_WS('-', 'a', 'b')", "hive": "CONCAT_WS('-', 'a', 'b')",
"spark": "CONCAT_WS('-', 'a', 'b')", "spark": "CONCAT_WS('-', 'a', 'b')",
"trino": "CONCAT_WS('-', 'a', 'b')", "trino": "CONCAT_WS('-', CAST('a' AS VARCHAR), CAST('b' AS VARCHAR))",
}, },
) )
self.validate_all( self.validate_all(
"CONCAT_WS('-', x)", "CONCAT_WS('-', x)",
write={ write={
"clickhouse": "CONCAT_WS('-', x)",
"duckdb": "CONCAT_WS('-', x)", "duckdb": "CONCAT_WS('-', x)",
"hive": "CONCAT_WS('-', x)", "hive": "CONCAT_WS('-', x)",
"presto": "CONCAT_WS('-', x)", "presto": "CONCAT_WS('-', CAST(x AS VARCHAR))",
"spark": "CONCAT_WS('-', x)", "spark": "CONCAT_WS('-', x)",
"trino": "CONCAT_WS('-', x)", "trino": "CONCAT_WS('-', CAST(x AS VARCHAR))",
}, },
) )
self.validate_all( self.validate_all(
"CONCAT(a)", "CONCAT(a)",
write={ write={
"clickhouse": "a", "clickhouse": "CONCAT(a)",
"presto": "a", "presto": "CAST(a AS VARCHAR)",
"trino": "a", "trino": "CAST(a AS VARCHAR)",
"tsql": "a", "tsql": "a",
}, },
) )
self.validate_all( self.validate_all(
"COALESCE(CAST(a AS TEXT), '')", "CONCAT(COALESCE(a, ''))",
read={ read={
"drill": "CONCAT(a)", "drill": "CONCAT(a)",
"duckdb": "CONCAT(a)", "duckdb": "CONCAT(a)",
@ -1442,6 +1486,76 @@ class TestDialect(Validator):
"spark": "FILTER(the_array, x -> x > 0)", "spark": "FILTER(the_array, x -> x > 0)",
}, },
) )
self.validate_all(
"a / b",
write={
"bigquery": "a / b",
"clickhouse": "a / b",
"databricks": "a / b",
"duckdb": "a / b",
"hive": "a / b",
"mysql": "a / b",
"oracle": "a / b",
"snowflake": "a / b",
"spark": "a / b",
"starrocks": "a / b",
"drill": "CAST(a AS DOUBLE) / b",
"postgres": "CAST(a AS DOUBLE PRECISION) / b",
"presto": "CAST(a AS DOUBLE) / b",
"redshift": "CAST(a AS DOUBLE PRECISION) / b",
"sqlite": "CAST(a AS REAL) / b",
"teradata": "CAST(a AS DOUBLE) / b",
"trino": "CAST(a AS DOUBLE) / b",
"tsql": "CAST(a AS FLOAT) / b",
},
)
def test_typeddiv(self):
typed_div = exp.Div(this=exp.column("a"), expression=exp.column("b"), typed=True)
div = exp.Div(this=exp.column("a"), expression=exp.column("b"))
typed_div_dialect = "presto"
div_dialect = "hive"
INT = exp.DataType.Type.INT
FLOAT = exp.DataType.Type.FLOAT
for expression, types, dialect, expected in [
(typed_div, (None, None), typed_div_dialect, "a / b"),
(typed_div, (None, None), div_dialect, "a / b"),
(div, (None, None), typed_div_dialect, "CAST(a AS DOUBLE) / b"),
(div, (None, None), div_dialect, "a / b"),
(typed_div, (INT, INT), typed_div_dialect, "a / b"),
(typed_div, (INT, INT), div_dialect, "CAST(a / b AS BIGINT)"),
(div, (INT, INT), typed_div_dialect, "CAST(a AS DOUBLE) / b"),
(div, (INT, INT), div_dialect, "a / b"),
(typed_div, (FLOAT, FLOAT), typed_div_dialect, "a / b"),
(typed_div, (FLOAT, FLOAT), div_dialect, "a / b"),
(div, (FLOAT, FLOAT), typed_div_dialect, "a / b"),
(div, (FLOAT, FLOAT), div_dialect, "a / b"),
(typed_div, (INT, FLOAT), typed_div_dialect, "a / b"),
(typed_div, (INT, FLOAT), div_dialect, "a / b"),
(div, (INT, FLOAT), typed_div_dialect, "a / b"),
(div, (INT, FLOAT), div_dialect, "a / b"),
]:
with self.subTest(f"{expression.__class__.__name__} {types} {dialect} -> {expected}"):
expression = expression.copy()
expression.left.type = types[0]
expression.right.type = types[1]
self.assertEqual(expected, expression.sql(dialect=dialect))
def test_safediv(self):
safe_div = exp.Div(this=exp.column("a"), expression=exp.column("b"), safe=True)
div = exp.Div(this=exp.column("a"), expression=exp.column("b"))
safe_div_dialect = "mysql"
div_dialect = "snowflake"
for expression, dialect, expected in [
(safe_div, safe_div_dialect, "a / b"),
(safe_div, div_dialect, "a / NULLIF(b, 0)"),
(div, safe_div_dialect, "a / b"),
(div, div_dialect, "a / b"),
]:
with self.subTest(f"{expression.__class__.__name__} {dialect} -> {expected}"):
self.assertEqual(expected, expression.sql(dialect=dialect))
def test_limit(self): def test_limit(self):
self.validate_all( self.validate_all(
@ -1547,7 +1661,7 @@ class TestDialect(Validator):
"CREATE TABLE t (b1 BINARY, b2 BINARY(1024), c1 TEXT, c2 TEXT(1024))", "CREATE TABLE t (b1 BINARY, b2 BINARY(1024), c1 TEXT, c2 TEXT(1024))",
write={ write={
"duckdb": "CREATE TABLE t (b1 BLOB, b2 BLOB(1024), c1 TEXT, c2 TEXT(1024))", "duckdb": "CREATE TABLE t (b1 BLOB, b2 BLOB(1024), c1 TEXT, c2 TEXT(1024))",
"hive": "CREATE TABLE t (b1 BINARY, b2 BINARY(1024), c1 STRING, c2 STRING(1024))", "hive": "CREATE TABLE t (b1 BINARY, b2 BINARY(1024), c1 STRING, c2 VARCHAR(1024))",
"oracle": "CREATE TABLE t (b1 BLOB, b2 BLOB(1024), c1 CLOB, c2 CLOB(1024))", "oracle": "CREATE TABLE t (b1 BLOB, b2 BLOB(1024), c1 CLOB, c2 CLOB(1024))",
"postgres": "CREATE TABLE t (b1 BYTEA, b2 BYTEA(1024), c1 TEXT, c2 TEXT(1024))", "postgres": "CREATE TABLE t (b1 BYTEA, b2 BYTEA(1024), c1 TEXT, c2 TEXT(1024))",
"sqlite": "CREATE TABLE t (b1 BLOB, b2 BLOB(1024), c1 TEXT, c2 TEXT(1024))", "sqlite": "CREATE TABLE t (b1 BLOB, b2 BLOB(1024), c1 TEXT, c2 TEXT(1024))",
@ -1864,7 +1978,7 @@ SELECT
write={ write={
"bigquery": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "bigquery": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"clickhouse": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "clickhouse": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"databricks": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "databricks": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
"duckdb": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "duckdb": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"hive": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq", "hive": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
"mysql": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "mysql": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
@ -1872,11 +1986,11 @@ SELECT
"presto": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "presto": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"redshift": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "redshift": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"snowflake": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "snowflake": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"spark": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "spark": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
"spark2": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq", "spark2": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
"sqlite": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "sqlite": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"trino": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq", "trino": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"tsql": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq", "tsql": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c AS c FROM t) AS subq",
}, },
) )
self.validate_all( self.validate_all(
@ -1885,13 +1999,60 @@ SELECT
"bigquery": "SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2", "bigquery": "SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2",
"duckdb": "SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2", "duckdb": "SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2",
"hive": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT * FROM (SELECT c FROM t) AS subq1) AS subq2", "hive": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT * FROM (SELECT c FROM t) AS subq1) AS subq2",
"tsql": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT * FROM (SELECT c FROM t) AS subq1) AS subq2", "tsql": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT * FROM (SELECT c AS c FROM t) AS subq1) AS subq2",
}, },
) )
self.validate_all( self.validate_all(
"WITH t1(x) AS (SELECT 1) SELECT * FROM (WITH t2(y) AS (SELECT 2) SELECT y FROM t2) AS subq", "WITH t1(x) AS (SELECT 1) SELECT * FROM (WITH t2(y) AS (SELECT 2) SELECT y FROM t2) AS subq",
write={ write={
"duckdb": "WITH t1(x) AS (SELECT 1) SELECT * FROM (WITH t2(y) AS (SELECT 2) SELECT y FROM t2) AS subq", "duckdb": "WITH t1(x) AS (SELECT 1) SELECT * FROM (WITH t2(y) AS (SELECT 2) SELECT y FROM t2) AS subq",
"tsql": "WITH t1(x) AS (SELECT 1), t2(y) AS (SELECT 2) SELECT * FROM (SELECT y FROM t2) AS subq", "tsql": "WITH t1(x) AS (SELECT 1), t2(y) AS (SELECT 2) SELECT * FROM (SELECT y AS y FROM t2) AS subq",
}, },
) )
def test_unsupported_null_ordering(self):
# We'll transpile a portable query from the following dialects to MySQL / T-SQL, which
# both treat NULLs as small values, so the expected output queries should be equivalent
with_last_nulls = "duckdb"
with_small_nulls = "spark"
with_large_nulls = "postgres"
sql = "SELECT * FROM t ORDER BY c"
sql_nulls_last = "SELECT * FROM t ORDER BY CASE WHEN c IS NULL THEN 1 ELSE 0 END, c"
sql_nulls_first = "SELECT * FROM t ORDER BY CASE WHEN c IS NULL THEN 1 ELSE 0 END DESC, c"
for read_dialect, desc, nulls_first, expected_sql in (
(with_last_nulls, False, None, sql_nulls_last),
(with_last_nulls, True, None, sql),
(with_last_nulls, False, True, sql),
(with_last_nulls, True, True, sql_nulls_first),
(with_last_nulls, False, False, sql_nulls_last),
(with_last_nulls, True, False, sql),
(with_small_nulls, False, None, sql),
(with_small_nulls, True, None, sql),
(with_small_nulls, False, True, sql),
(with_small_nulls, True, True, sql_nulls_first),
(with_small_nulls, False, False, sql_nulls_last),
(with_small_nulls, True, False, sql),
(with_large_nulls, False, None, sql_nulls_last),
(with_large_nulls, True, None, sql_nulls_first),
(with_large_nulls, False, True, sql),
(with_large_nulls, True, True, sql_nulls_first),
(with_large_nulls, False, False, sql_nulls_last),
(with_large_nulls, True, False, sql),
):
with self.subTest(
f"read: {read_dialect}, descending: {desc}, nulls first: {nulls_first}"
):
sort_order = " DESC" if desc else ""
null_order = (
" NULLS FIRST"
if nulls_first
else (" NULLS LAST" if nulls_first is not None else "")
)
expected_sql = f"{expected_sql}{sort_order}"
expression = parse_one(f"{sql}{sort_order}{null_order}", read=read_dialect)
self.assertEqual(expression.sql(dialect="mysql"), expected_sql)
self.assertEqual(expression.sql(dialect="tsql"), expected_sql)

124
tests/dialects/test_duckdb.py
View file

@ -1,4 +1,5 @@
from sqlglot import ErrorLevel, UnsupportedError, exp, parse_one, transpile from sqlglot import ErrorLevel, UnsupportedError, exp, parse_one, transpile
from sqlglot.helper import logger as helper_logger
from tests.dialects.test_dialect import Validator from tests.dialects.test_dialect import Validator
@ -71,7 +72,7 @@ class TestDuckDB(Validator):
"SELECT UNNEST(ARRAY[1, 2, 3]), UNNEST(ARRAY[4, 5]), UNNEST(ARRAY[6])", "SELECT UNNEST(ARRAY[1, 2, 3]), UNNEST(ARRAY[4, 5]), UNNEST(ARRAY[6])",
write={ write={
"bigquery": "SELECT IF(pos = pos_2, col, NULL) AS col, IF(pos = pos_3, col_2, NULL) AS col_2, IF(pos = pos_4, col_3, NULL) AS col_3 FROM UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH([1, 2, 3]), ARRAY_LENGTH([4, 5]), ARRAY_LENGTH([6])) - 1)) AS pos CROSS JOIN UNNEST([1, 2, 3]) AS col WITH OFFSET AS pos_2 CROSS JOIN UNNEST([4, 5]) AS col_2 WITH OFFSET AS pos_3 CROSS JOIN UNNEST([6]) AS col_3 WITH OFFSET AS pos_4 WHERE ((pos = pos_2 OR (pos > (ARRAY_LENGTH([1, 2, 3]) - 1) AND pos_2 = (ARRAY_LENGTH([1, 2, 3]) - 1))) AND (pos = pos_3 OR (pos > (ARRAY_LENGTH([4, 5]) - 1) AND pos_3 = (ARRAY_LENGTH([4, 5]) - 1)))) AND (pos = pos_4 OR (pos > (ARRAY_LENGTH([6]) - 1) AND pos_4 = (ARRAY_LENGTH([6]) - 1)))", "bigquery": "SELECT IF(pos = pos_2, col, NULL) AS col, IF(pos = pos_3, col_2, NULL) AS col_2, IF(pos = pos_4, col_3, NULL) AS col_3 FROM UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH([1, 2, 3]), ARRAY_LENGTH([4, 5]), ARRAY_LENGTH([6])) - 1)) AS pos CROSS JOIN UNNEST([1, 2, 3]) AS col WITH OFFSET AS pos_2 CROSS JOIN UNNEST([4, 5]) AS col_2 WITH OFFSET AS pos_3 CROSS JOIN UNNEST([6]) AS col_3 WITH OFFSET AS pos_4 WHERE ((pos = pos_2 OR (pos > (ARRAY_LENGTH([1, 2, 3]) - 1) AND pos_2 = (ARRAY_LENGTH([1, 2, 3]) - 1))) AND (pos = pos_3 OR (pos > (ARRAY_LENGTH([4, 5]) - 1) AND pos_3 = (ARRAY_LENGTH([4, 5]) - 1)))) AND (pos = pos_4 OR (pos > (ARRAY_LENGTH([6]) - 1) AND pos_4 = (ARRAY_LENGTH([6]) - 1)))",
"presto": "SELECT IF(pos = pos_2, col) AS col, IF(pos = pos_3, col_2) AS col_2, IF(pos = pos_4, col_3) AS col_3 FROM UNNEST(SEQUENCE(1, GREATEST(CARDINALITY(ARRAY[1, 2, 3]), CARDINALITY(ARRAY[4, 5]), CARDINALITY(ARRAY[6])))) AS _u(pos) CROSS JOIN UNNEST(ARRAY[1, 2, 3]) WITH ORDINALITY AS _u_2(col, pos_2) CROSS JOIN UNNEST(ARRAY[4, 5]) WITH ORDINALITY AS _u_3(col_2, pos_3) CROSS JOIN UNNEST(ARRAY[6]) WITH ORDINALITY AS _u_4(col_3, pos_4) WHERE ((pos = pos_2 OR (pos > CARDINALITY(ARRAY[1, 2, 3]) AND pos_2 = CARDINALITY(ARRAY[1, 2, 3]))) AND (pos = pos_3 OR (pos > CARDINALITY(ARRAY[4, 5]) AND pos_3 = CARDINALITY(ARRAY[4, 5])))) AND (pos = pos_4 OR (pos > CARDINALITY(ARRAY[6]) AND pos_4 = CARDINALITY(ARRAY[6])))", "presto": "SELECT IF(_u.pos = _u_2.pos_2, _u_2.col) AS col, IF(_u.pos = _u_3.pos_3, _u_3.col_2) AS col_2, IF(_u.pos = _u_4.pos_4, _u_4.col_3) AS col_3 FROM UNNEST(SEQUENCE(1, GREATEST(CARDINALITY(ARRAY[1, 2, 3]), CARDINALITY(ARRAY[4, 5]), CARDINALITY(ARRAY[6])))) AS _u(pos) CROSS JOIN UNNEST(ARRAY[1, 2, 3]) WITH ORDINALITY AS _u_2(col, pos_2) CROSS JOIN UNNEST(ARRAY[4, 5]) WITH ORDINALITY AS _u_3(col_2, pos_3) CROSS JOIN UNNEST(ARRAY[6]) WITH ORDINALITY AS _u_4(col_3, pos_4) WHERE ((_u.pos = _u_2.pos_2 OR (_u.pos > CARDINALITY(ARRAY[1, 2, 3]) AND _u_2.pos_2 = CARDINALITY(ARRAY[1, 2, 3]))) AND (_u.pos = _u_3.pos_3 OR (_u.pos > CARDINALITY(ARRAY[4, 5]) AND _u_3.pos_3 = CARDINALITY(ARRAY[4, 5])))) AND (_u.pos = _u_4.pos_4 OR (_u.pos > CARDINALITY(ARRAY[6]) AND _u_4.pos_4 = CARDINALITY(ARRAY[6])))",
}, },
) )
@ -79,7 +80,7 @@ class TestDuckDB(Validator):
"SELECT UNNEST(ARRAY[1, 2, 3]), UNNEST(ARRAY[4, 5]), UNNEST(ARRAY[6]) FROM x", "SELECT UNNEST(ARRAY[1, 2, 3]), UNNEST(ARRAY[4, 5]), UNNEST(ARRAY[6]) FROM x",
write={ write={
"bigquery": "SELECT IF(pos = pos_2, col, NULL) AS col, IF(pos = pos_3, col_2, NULL) AS col_2, IF(pos = pos_4, col_3, NULL) AS col_3 FROM x, UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH([1, 2, 3]), ARRAY_LENGTH([4, 5]), ARRAY_LENGTH([6])) - 1)) AS pos CROSS JOIN UNNEST([1, 2, 3]) AS col WITH OFFSET AS pos_2 CROSS JOIN UNNEST([4, 5]) AS col_2 WITH OFFSET AS pos_3 CROSS JOIN UNNEST([6]) AS col_3 WITH OFFSET AS pos_4 WHERE ((pos = pos_2 OR (pos > (ARRAY_LENGTH([1, 2, 3]) - 1) AND pos_2 = (ARRAY_LENGTH([1, 2, 3]) - 1))) AND (pos = pos_3 OR (pos > (ARRAY_LENGTH([4, 5]) - 1) AND pos_3 = (ARRAY_LENGTH([4, 5]) - 1)))) AND (pos = pos_4 OR (pos > (ARRAY_LENGTH([6]) - 1) AND pos_4 = (ARRAY_LENGTH([6]) - 1)))", "bigquery": "SELECT IF(pos = pos_2, col, NULL) AS col, IF(pos = pos_3, col_2, NULL) AS col_2, IF(pos = pos_4, col_3, NULL) AS col_3 FROM x, UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH([1, 2, 3]), ARRAY_LENGTH([4, 5]), ARRAY_LENGTH([6])) - 1)) AS pos CROSS JOIN UNNEST([1, 2, 3]) AS col WITH OFFSET AS pos_2 CROSS JOIN UNNEST([4, 5]) AS col_2 WITH OFFSET AS pos_3 CROSS JOIN UNNEST([6]) AS col_3 WITH OFFSET AS pos_4 WHERE ((pos = pos_2 OR (pos > (ARRAY_LENGTH([1, 2, 3]) - 1) AND pos_2 = (ARRAY_LENGTH([1, 2, 3]) - 1))) AND (pos = pos_3 OR (pos > (ARRAY_LENGTH([4, 5]) - 1) AND pos_3 = (ARRAY_LENGTH([4, 5]) - 1)))) AND (pos = pos_4 OR (pos > (ARRAY_LENGTH([6]) - 1) AND pos_4 = (ARRAY_LENGTH([6]) - 1)))",
"presto": "SELECT IF(pos = pos_2, col) AS col, IF(pos = pos_3, col_2) AS col_2, IF(pos = pos_4, col_3) AS col_3 FROM x, UNNEST(SEQUENCE(1, GREATEST(CARDINALITY(ARRAY[1, 2, 3]), CARDINALITY(ARRAY[4, 5]), CARDINALITY(ARRAY[6])))) AS _u(pos) CROSS JOIN UNNEST(ARRAY[1, 2, 3]) WITH ORDINALITY AS _u_2(col, pos_2) CROSS JOIN UNNEST(ARRAY[4, 5]) WITH ORDINALITY AS _u_3(col_2, pos_3) CROSS JOIN UNNEST(ARRAY[6]) WITH ORDINALITY AS _u_4(col_3, pos_4) WHERE ((pos = pos_2 OR (pos > CARDINALITY(ARRAY[1, 2, 3]) AND pos_2 = CARDINALITY(ARRAY[1, 2, 3]))) AND (pos = pos_3 OR (pos > CARDINALITY(ARRAY[4, 5]) AND pos_3 = CARDINALITY(ARRAY[4, 5])))) AND (pos = pos_4 OR (pos > CARDINALITY(ARRAY[6]) AND pos_4 = CARDINALITY(ARRAY[6])))", "presto": "SELECT IF(_u.pos = _u_2.pos_2, _u_2.col) AS col, IF(_u.pos = _u_3.pos_3, _u_3.col_2) AS col_2, IF(_u.pos = _u_4.pos_4, _u_4.col_3) AS col_3 FROM x, UNNEST(SEQUENCE(1, GREATEST(CARDINALITY(ARRAY[1, 2, 3]), CARDINALITY(ARRAY[4, 5]), CARDINALITY(ARRAY[6])))) AS _u(pos) CROSS JOIN UNNEST(ARRAY[1, 2, 3]) WITH ORDINALITY AS _u_2(col, pos_2) CROSS JOIN UNNEST(ARRAY[4, 5]) WITH ORDINALITY AS _u_3(col_2, pos_3) CROSS JOIN UNNEST(ARRAY[6]) WITH ORDINALITY AS _u_4(col_3, pos_4) WHERE ((_u.pos = _u_2.pos_2 OR (_u.pos > CARDINALITY(ARRAY[1, 2, 3]) AND _u_2.pos_2 = CARDINALITY(ARRAY[1, 2, 3]))) AND (_u.pos = _u_3.pos_3 OR (_u.pos > CARDINALITY(ARRAY[4, 5]) AND _u_3.pos_3 = CARDINALITY(ARRAY[4, 5])))) AND (_u.pos = _u_4.pos_4 OR (_u.pos > CARDINALITY(ARRAY[6]) AND _u_4.pos_4 = CARDINALITY(ARRAY[6])))",
}, },
) )
self.validate_all( self.validate_all(
@ -96,7 +97,6 @@ class TestDuckDB(Validator):
) )
self.validate_identity("SELECT i FROM RANGE(5) AS _(i) ORDER BY i ASC") self.validate_identity("SELECT i FROM RANGE(5) AS _(i) ORDER BY i ASC")
self.validate_identity("[x.STRING_SPLIT(' ')[1] FOR x IN ['1', '2', 3] IF x.CONTAINS('1')]")
self.validate_identity("INSERT INTO x BY NAME SELECT 1 AS y") self.validate_identity("INSERT INTO x BY NAME SELECT 1 AS y")
self.validate_identity("SELECT 1 AS x UNION ALL BY NAME SELECT 2 AS x") self.validate_identity("SELECT 1 AS x UNION ALL BY NAME SELECT 2 AS x")
self.validate_identity("SELECT SUM(x) FILTER (x = 1)", "SELECT SUM(x) FILTER(WHERE x = 1)") self.validate_identity("SELECT SUM(x) FILTER (x = 1)", "SELECT SUM(x) FILTER(WHERE x = 1)")
@ -109,6 +109,10 @@ class TestDuckDB(Validator):
parse_one("a // b", read="duckdb").assert_is(exp.IntDiv).sql(dialect="duckdb"), "a // b" parse_one("a // b", read="duckdb").assert_is(exp.IntDiv).sql(dialect="duckdb"), "a // b"
) )
self.validate_identity("SELECT EPOCH_MS(10) AS t")
self.validate_identity("SELECT MAKE_TIMESTAMP(10) AS t")
self.validate_identity("SELECT TO_TIMESTAMP(10) AS t")
self.validate_identity("SELECT UNNEST(column, recursive := TRUE) FROM table")
self.validate_identity("VAR_POP(a)") self.validate_identity("VAR_POP(a)")
self.validate_identity("SELECT * FROM foo ASOF LEFT JOIN bar ON a = b") self.validate_identity("SELECT * FROM foo ASOF LEFT JOIN bar ON a = b")
self.validate_identity("PIVOT Cities ON Year USING SUM(Population)") self.validate_identity("PIVOT Cities ON Year USING SUM(Population)")
@ -151,11 +155,18 @@ class TestDuckDB(Validator):
self.validate_all("0x1010", write={"": "0 AS x1010"}) self.validate_all("0x1010", write={"": "0 AS x1010"})
self.validate_all("x ~ y", write={"duckdb": "REGEXP_MATCHES(x, y)"}) self.validate_all("x ~ y", write={"duckdb": "REGEXP_MATCHES(x, y)"})
self.validate_all("SELECT * FROM 'x.y'", write={"duckdb": 'SELECT * FROM "x.y"'}) self.validate_all("SELECT * FROM 'x.y'", write={"duckdb": 'SELECT * FROM "x.y"'})
self.validate_all(
"SELECT * FROM produce PIVOT(SUM(sales) FOR quarter IN ('Q1', 'Q2'))",
read={
"duckdb": "SELECT * FROM produce PIVOT(SUM(sales) FOR quarter IN ('Q1', 'Q2'))",
"snowflake": "SELECT * FROM produce PIVOT(SUM(produce.sales) FOR produce.quarter IN ('Q1', 'Q2'))",
},
)
self.validate_all( self.validate_all(
"SELECT UNNEST([1, 2, 3])", "SELECT UNNEST([1, 2, 3])",
write={ write={
"duckdb": "SELECT UNNEST([1, 2, 3])", "duckdb": "SELECT UNNEST([1, 2, 3])",
"snowflake": "SELECT IFF(pos = pos_2, col, NULL) AS col FROM (SELECT value FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (GREATEST(ARRAY_SIZE([1, 2, 3])) - 1) + 1)))) AS _u(pos) CROSS JOIN (SELECT value, index FROM TABLE(FLATTEN(INPUT => [1, 2, 3]))) AS _u_2(col, pos_2) WHERE pos = pos_2 OR (pos > (ARRAY_SIZE([1, 2, 3]) - 1) AND pos_2 = (ARRAY_SIZE([1, 2, 3]) - 1))", "snowflake": "SELECT IFF(_u.pos = _u_2.pos_2, _u_2.col, NULL) AS col FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (GREATEST(ARRAY_SIZE([1, 2, 3])) - 1) + 1))) AS _u(seq, key, path, index, pos, this) CROSS JOIN TABLE(FLATTEN(INPUT => [1, 2, 3])) AS _u_2(seq, key, path, pos_2, col, this) WHERE _u.pos = _u_2.pos_2 OR (_u.pos > (ARRAY_SIZE([1, 2, 3]) - 1) AND _u_2.pos_2 = (ARRAY_SIZE([1, 2, 3]) - 1))",
}, },
) )
self.validate_all( self.validate_all(
@ -355,14 +366,14 @@ class TestDuckDB(Validator):
"STRUCT_PACK(x := 1, y := '2')", "STRUCT_PACK(x := 1, y := '2')",
write={ write={
"duckdb": "{'x': 1, 'y': '2'}", "duckdb": "{'x': 1, 'y': '2'}",
"spark": "STRUCT(x = 1, y = '2')", "spark": "STRUCT(1 AS x, '2' AS y)",
}, },
) )
self.validate_all( self.validate_all(
"STRUCT_PACK(key1 := 'value1', key2 := 42)", "STRUCT_PACK(key1 := 'value1', key2 := 42)",
write={ write={
"duckdb": "{'key1': 'value1', 'key2': 42}", "duckdb": "{'key1': 'value1', 'key2': 42}",
"spark": "STRUCT(key1 = 'value1', key2 = 42)", "spark": "STRUCT('value1' AS key1, 42 AS key2)",
}, },
) )
self.validate_all( self.validate_all(
@ -440,6 +451,16 @@ class TestDuckDB(Validator):
"hive": "SELECT DATE_ADD(TO_DATE(x), 1)", "hive": "SELECT DATE_ADD(TO_DATE(x), 1)",
}, },
) )
self.validate_all(
"SELECT CAST('2018-01-01 00:00:00' AS DATE) + INTERVAL 3 DAY",
read={
"hive": "SELECT DATE_ADD('2018-01-01 00:00:00', 3)",
},
write={
"duckdb": "SELECT CAST('2018-01-01 00:00:00' AS DATE) + INTERVAL '3' DAY",
"hive": "SELECT CAST('2018-01-01 00:00:00' AS DATE) + INTERVAL '3' DAY",
},
)
self.validate_all( self.validate_all(
"SELECT CAST('2020-05-06' AS DATE) - INTERVAL 5 DAY", "SELECT CAST('2020-05-06' AS DATE) - INTERVAL 5 DAY",
read={"bigquery": "SELECT DATE_SUB(CAST('2020-05-06' AS DATE), INTERVAL 5 DAY)"}, read={"bigquery": "SELECT DATE_SUB(CAST('2020-05-06' AS DATE), INTERVAL 5 DAY)"},
@ -483,6 +504,35 @@ class TestDuckDB(Validator):
self.validate_identity("SELECT ISNAN(x)") self.validate_identity("SELECT ISNAN(x)")
def test_array_index(self):
with self.assertLogs(helper_logger) as cm:
self.validate_all(
"SELECT some_arr[1] AS first FROM blah",
read={
"bigquery": "SELECT some_arr[0] AS first FROM blah",
},
write={
"bigquery": "SELECT some_arr[0] AS first FROM blah",
"duckdb": "SELECT some_arr[1] AS first FROM blah",
"presto": "SELECT some_arr[1] AS first FROM blah",
},
)
self.validate_identity(
"[x.STRING_SPLIT(' ')[1] FOR x IN ['1', '2', 3] IF x.CONTAINS('1')]"
)
self.assertEqual(
cm.output,
[
"WARNING:sqlglot:Applying array index offset (-1)",
"WARNING:sqlglot:Applying array index offset (1)",
"WARNING:sqlglot:Applying array index offset (1)",
"WARNING:sqlglot:Applying array index offset (1)",
"WARNING:sqlglot:Applying array index offset (-1)",
"WARNING:sqlglot:Applying array index offset (1)",
],
)
def test_time(self): def test_time(self):
self.validate_identity("SELECT CURRENT_DATE") self.validate_identity("SELECT CURRENT_DATE")
self.validate_identity("SELECT CURRENT_TIMESTAMP") self.validate_identity("SELECT CURRENT_TIMESTAMP")
@ -533,16 +583,16 @@ class TestDuckDB(Validator):
self.validate_all( self.validate_all(
"EPOCH_MS(x)", "EPOCH_MS(x)",
write={ write={
"bigquery": "UNIX_TO_TIME(x / 1000)", "bigquery": "TIMESTAMP_MILLIS(x)",
"duckdb": "TO_TIMESTAMP(x / 1000)", "duckdb": "EPOCH_MS(x)",
"presto": "FROM_UNIXTIME(x / 1000)", "presto": "FROM_UNIXTIME(CAST(x AS DOUBLE) / 1000)",
"spark": "CAST(FROM_UNIXTIME(x / 1000) AS TIMESTAMP)", "spark": "TIMESTAMP_MILLIS(x)",
}, },
) )
self.validate_all( self.validate_all(
"STRFTIME(x, '%y-%-m-%S')", "STRFTIME(x, '%y-%-m-%S')",
write={ write={
"bigquery": "TIME_TO_STR(x, '%y-%-m-%S')", "bigquery": "FORMAT_DATE('%y-%-m-%S', x)",
"duckdb": "STRFTIME(x, '%y-%-m-%S')", "duckdb": "STRFTIME(x, '%y-%-m-%S')",
"postgres": "TO_CHAR(x, 'YY-FMMM-SS')", "postgres": "TO_CHAR(x, 'YY-FMMM-SS')",
"presto": "DATE_FORMAT(x, '%y-%c-%s')", "presto": "DATE_FORMAT(x, '%y-%c-%s')",
@ -552,6 +602,7 @@ class TestDuckDB(Validator):
self.validate_all( self.validate_all(
"STRFTIME(x, '%Y-%m-%d %H:%M:%S')", "STRFTIME(x, '%Y-%m-%d %H:%M:%S')",
write={ write={
"bigquery": "FORMAT_DATE('%Y-%m-%d %H:%M:%S', x)",
"duckdb": "STRFTIME(x, '%Y-%m-%d %H:%M:%S')", "duckdb": "STRFTIME(x, '%Y-%m-%d %H:%M:%S')",
"presto": "DATE_FORMAT(x, '%Y-%m-%d %T')", "presto": "DATE_FORMAT(x, '%Y-%m-%d %T')",
"hive": "DATE_FORMAT(x, 'yyyy-MM-dd HH:mm:ss')", "hive": "DATE_FORMAT(x, 'yyyy-MM-dd HH:mm:ss')",
@ -570,7 +621,7 @@ class TestDuckDB(Validator):
self.validate_all( self.validate_all(
"TO_TIMESTAMP(x)", "TO_TIMESTAMP(x)",
write={ write={
"bigquery": "UNIX_TO_TIME(x)", "bigquery": "TIMESTAMP_SECONDS(x)",
"duckdb": "TO_TIMESTAMP(x)", "duckdb": "TO_TIMESTAMP(x)",
"presto": "FROM_UNIXTIME(x)", "presto": "FROM_UNIXTIME(x)",
"hive": "FROM_UNIXTIME(x)", "hive": "FROM_UNIXTIME(x)",
@ -651,22 +702,25 @@ class TestDuckDB(Validator):
"CAST(ROW(1, ROW(1)) AS STRUCT(number BIGINT, row STRUCT(number BIGINT)))" "CAST(ROW(1, ROW(1)) AS STRUCT(number BIGINT, row STRUCT(number BIGINT)))"
) )
self.validate_all("CAST(x AS NUMERIC(1, 2))", write={"duckdb": "CAST(x AS DECIMAL(1, 2))"}) self.validate_identity("CAST(x AS INT64)", "CAST(x AS BIGINT)")
self.validate_all("CAST(x AS HUGEINT)", write={"duckdb": "CAST(x AS INT128)"}) self.validate_identity("CAST(x AS INT32)", "CAST(x AS INT)")
self.validate_all("CAST(x AS CHAR)", write={"duckdb": "CAST(x AS TEXT)"}) self.validate_identity("CAST(x AS INT16)", "CAST(x AS SMALLINT)")
self.validate_all("CAST(x AS BPCHAR)", write={"duckdb": "CAST(x AS TEXT)"}) self.validate_identity("CAST(x AS NUMERIC(1, 2))", "CAST(x AS DECIMAL(1, 2))")
self.validate_all("CAST(x AS STRING)", write={"duckdb": "CAST(x AS TEXT)"}) self.validate_identity("CAST(x AS HUGEINT)", "CAST(x AS INT128)")
self.validate_all("CAST(x AS INT1)", write={"duckdb": "CAST(x AS TINYINT)"}) self.validate_identity("CAST(x AS CHAR)", "CAST(x AS TEXT)")
self.validate_all("CAST(x AS FLOAT4)", write={"duckdb": "CAST(x AS REAL)"}) self.validate_identity("CAST(x AS BPCHAR)", "CAST(x AS TEXT)")
self.validate_all("CAST(x AS FLOAT)", write={"duckdb": "CAST(x AS REAL)"}) self.validate_identity("CAST(x AS STRING)", "CAST(x AS TEXT)")
self.validate_all("CAST(x AS INT4)", write={"duckdb": "CAST(x AS INT)"}) self.validate_identity("CAST(x AS INT1)", "CAST(x AS TINYINT)")
self.validate_all("CAST(x AS INTEGER)", write={"duckdb": "CAST(x AS INT)"}) self.validate_identity("CAST(x AS FLOAT4)", "CAST(x AS REAL)")
self.validate_all("CAST(x AS SIGNED)", write={"duckdb": "CAST(x AS INT)"}) self.validate_identity("CAST(x AS FLOAT)", "CAST(x AS REAL)")
self.validate_all("CAST(x AS BLOB)", write={"duckdb": "CAST(x AS BLOB)"}) self.validate_identity("CAST(x AS INT4)", "CAST(x AS INT)")
self.validate_all("CAST(x AS BYTEA)", write={"duckdb": "CAST(x AS BLOB)"}) self.validate_identity("CAST(x AS INTEGER)", "CAST(x AS INT)")
self.validate_all("CAST(x AS BINARY)", write={"duckdb": "CAST(x AS BLOB)"}) self.validate_identity("CAST(x AS SIGNED)", "CAST(x AS INT)")
self.validate_all("CAST(x AS VARBINARY)", write={"duckdb": "CAST(x AS BLOB)"}) self.validate_identity("CAST(x AS BLOB)", "CAST(x AS BLOB)")
self.validate_all("CAST(x AS LOGICAL)", write={"duckdb": "CAST(x AS BOOLEAN)"}) self.validate_identity("CAST(x AS BYTEA)", "CAST(x AS BLOB)")
self.validate_identity("CAST(x AS BINARY)", "CAST(x AS BLOB)")
self.validate_identity("CAST(x AS VARBINARY)", "CAST(x AS BLOB)")
self.validate_identity("CAST(x AS LOGICAL)", "CAST(x AS BOOLEAN)")
self.validate_all( self.validate_all(
"CAST(x AS NUMERIC)", "CAST(x AS NUMERIC)",
write={ write={
@ -799,3 +853,17 @@ class TestDuckDB(Validator):
"duckdb": "SELECT CAST(w AS TIMESTAMP_S), CAST(x AS TIMESTAMP_MS), CAST(y AS TIMESTAMP), CAST(z AS TIMESTAMP_NS)", "duckdb": "SELECT CAST(w AS TIMESTAMP_S), CAST(x AS TIMESTAMP_MS), CAST(y AS TIMESTAMP), CAST(z AS TIMESTAMP_NS)",
}, },
) )
def test_isnan(self):
self.validate_all(
"ISNAN(x)",
read={"bigquery": "IS_NAN(x)"},
write={"bigquery": "IS_NAN(x)", "duckdb": "ISNAN(x)"},
)
def test_isinf(self):
self.validate_all(
"ISINF(x)",
read={"bigquery": "IS_INF(x)"},
write={"bigquery": "IS_INF(x)", "duckdb": "ISINF(x)"},
)

Some files were not shown because too many files have changed in this diff Show more