dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Merging upstream version 23.13.1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-13 21:33:25 +01:00
parent 63a75c51ff
commit 64041d1d66
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
85 changed files with 53899 additions and 50390 deletions
Download patch file Download diff file Expand all files Collapse all files

83
CHANGELOG.md
View file

@ -1,6 +1,86 @@
Changelog Changelog
========= =========
## [v23.13.0] - 2024-05-03
### :boom: BREAKING CHANGES
- due to [`cc6259d`](https://github.com/tobymao/sqlglot/commit/cc6259de3d68831ded31bfb7fafe1ce654aa89dd) - Mark UDTF child scopes as ScopeType.SUBQUERY *(PR [#3390](https://github.com/tobymao/sqlglot/pull/3390) by [@VaggelisD](https://github.com/VaggelisD))*:
Mark UDTF child scopes as ScopeType.SUBQUERY (#3390)
- due to [`33bae9b`](https://github.com/tobymao/sqlglot/commit/33bae9b527b27f02dfafff3f45534f85aa9e0d9d) - get rid of superfluous "parameters" arg in RegexpReplace *(PR [#3394](https://github.com/tobymao/sqlglot/pull/3394) by [@georgesittas](https://github.com/georgesittas))*:
get rid of superfluous "parameters" arg in RegexpReplace (#3394)
- due to [`3768514`](https://github.com/tobymao/sqlglot/commit/3768514e3b2f256b69553e173b40f17180744ab0) - snowflake optional merge insert *(commit by [@tobymao](https://github.com/tobymao))*:
snowflake optional merge insert
- due to [`d1b4f1f`](https://github.com/tobymao/sqlglot/commit/d1b4f1f256cd772bec366d6bf13d9589e1fdfc4b) - Introducing TIMESTAMP_NTZ token and data type *(PR [#3386](https://github.com/tobymao/sqlglot/pull/3386) by [@VaggelisD](https://github.com/VaggelisD))*:
Introducing TIMESTAMP_NTZ token and data type (#3386)
### :sparkles: New Features
- [`d1b4f1f`](https://github.com/tobymao/sqlglot/commit/d1b4f1f256cd772bec366d6bf13d9589e1fdfc4b) - Introducing TIMESTAMP_NTZ token and data type *(PR [#3386](https://github.com/tobymao/sqlglot/pull/3386) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#3379](https://github.com/tobymao/sqlglot/issues/3379) opened by [@aersam](https://github.com/aersam)*
- [`16691f9`](https://github.com/tobymao/sqlglot/commit/16691f962822a132e233d61c2b67ec0fc3da51eb) - **prql**: add support for AGGREGATE *(PR [#3395](https://github.com/tobymao/sqlglot/pull/3395) by [@fool1280](https://github.com/fool1280))*
- [`534fb80`](https://github.com/tobymao/sqlglot/commit/534fb80462370b5236061472496c35a16e9bab4a) - **postgres**: add support for anonymos index DDL syntax *(PR [#3403](https://github.com/tobymao/sqlglot/pull/3403) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`a2afcca`](https://github.com/tobymao/sqlglot/commit/a2afccafd300939eaa5a3b075820f3bf8e8dcaac) - **mysql**: don't cast into invalid numeric/text types *(PR [#3375](https://github.com/tobymao/sqlglot/pull/3375) by [@georgesittas](https://github.com/georgesittas))*
- [`60b5c3b`](https://github.com/tobymao/sqlglot/commit/60b5c3b1b5dfb4aa00754f4b2473ad054b8dd14a) - **spark**: transpile presto TRY, fix JSON casting issue *(PR [#3376](https://github.com/tobymao/sqlglot/pull/3376) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3374](https://github.com/tobymao/sqlglot/issues/3374) opened by [@cploonker](https://github.com/cploonker)*
- [`3e8de71`](https://github.com/tobymao/sqlglot/commit/3e8de7124b735a6ab52971a3e51702c4e7b74be5) - **postgres**: allow FOR clause without FROM in SUBSTRING closes [#3377](https://github.com/tobymao/sqlglot/pull/3377) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`b2a7e55`](https://github.com/tobymao/sqlglot/commit/b2a7e550b25fd95eb0abba63228c9e285be168e0) - **optimizer**: Remove XOR from connector simplifications *(PR [#3380](https://github.com/tobymao/sqlglot/pull/3380) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3372](https://github.com/tobymao/sqlglot/issues/3372) opened by [@colincointe](https://github.com/colincointe)*
- [`477754c`](https://github.com/tobymao/sqlglot/commit/477754c72c47b6dc9dd01463b8f6fae6686cb1ac) - **trino**: bring back TRIM parsing *(PR [#3385](https://github.com/tobymao/sqlglot/pull/3385) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3384](https://github.com/tobymao/sqlglot/issues/3384) opened by [@dmelchor-stripe](https://github.com/dmelchor-stripe)*
- [`cc6259d`](https://github.com/tobymao/sqlglot/commit/cc6259de3d68831ded31bfb7fafe1ce654aa89dd) - **optimizer**: Mark UDTF child scopes as ScopeType.SUBQUERY *(PR [#3390](https://github.com/tobymao/sqlglot/pull/3390) by [@VaggelisD](https://github.com/VaggelisD))*
- [`0d23b20`](https://github.com/tobymao/sqlglot/commit/0d23b20352a8931adf8224d322da324b18e8282d) - allow joins in FROM expression parser *(PR [#3389](https://github.com/tobymao/sqlglot/pull/3389) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3387](https://github.com/tobymao/sqlglot/issues/3387) opened by [@MikeWallis42](https://github.com/MikeWallis42)*
- [`e7021df`](https://github.com/tobymao/sqlglot/commit/e7021df397a1dc5e726d1e391ef6428a3190856d) - **duckdb**: Preserve DATE_SUB roundtrip *(PR [#3382](https://github.com/tobymao/sqlglot/pull/3382) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3373](https://github.com/tobymao/sqlglot/issues/3373) opened by [@zergar](https://github.com/zergar)*
- [`641b296`](https://github.com/tobymao/sqlglot/commit/641b296017591b65ffc223d28b37e51886789ca7) - **postgres**: tokenize INT8 as BIGINT *(PR [#3392](https://github.com/tobymao/sqlglot/pull/3392) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3391](https://github.com/tobymao/sqlglot/issues/3391) opened by [@fuzi1996](https://github.com/fuzi1996)*
- [`33bae9b`](https://github.com/tobymao/sqlglot/commit/33bae9b527b27f02dfafff3f45534f85aa9e0d9d) - get rid of superfluous "parameters" arg in RegexpReplace *(PR [#3394](https://github.com/tobymao/sqlglot/pull/3394) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3393](https://github.com/tobymao/sqlglot/issues/3393) opened by [@rzykov](https://github.com/rzykov)*
- [`3768514`](https://github.com/tobymao/sqlglot/commit/3768514e3b2f256b69553e173b40f17180744ab0) - snowflake optional merge insert *(commit by [@tobymao](https://github.com/tobymao))*
- [`f44cd24`](https://github.com/tobymao/sqlglot/commit/f44cd248a82f5519afd0edba5112a499b804fe8f) - make generated constraint parsing more lenient fixes [#3397](https://github.com/tobymao/sqlglot/pull/3397) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`00ff877`](https://github.com/tobymao/sqlglot/commit/00ff87719ab4d6e3a407334c8d811366d0c7ead5) - **tsql**: quote hash sign as well for quoted temporary tables *(PR [#3401](https://github.com/tobymao/sqlglot/pull/3401) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3399](https://github.com/tobymao/sqlglot/issues/3399) opened by [@gforsyth](https://github.com/gforsyth)*
- [`84b7026`](https://github.com/tobymao/sqlglot/commit/84b7026e2fbc4e73c3b4c0c86cb764b95541841e) - **trino**: support for data type 'tdigest' *(PR [#3402](https://github.com/tobymao/sqlglot/pull/3402) by [@suryaiyer95](https://github.com/suryaiyer95))*
- [`24e1115`](https://github.com/tobymao/sqlglot/commit/24e1115c957d42a5511c1c428516e3ce5426cd88) - **trino|presto**: adding cast support for "hyperloglog" column type *(PR [#3405](https://github.com/tobymao/sqlglot/pull/3405) by [@uncledata](https://github.com/uncledata))*
## [v23.12.2] - 2024-04-30
### :sparkles: New Features
- [`d2a6f16`](https://github.com/tobymao/sqlglot/commit/d2a6f16c35cbe355932d0e0eab2fc6ba096d8a97) - COPY TO/FROM statement *(PR [#3359](https://github.com/tobymao/sqlglot/pull/3359) by [@VaggelisD](https://github.com/VaggelisD))*
- [`f034ea0`](https://github.com/tobymao/sqlglot/commit/f034ea0fdd7429bf6694e07b4aff06c665c10951) - **mysql**: Transpile TimestampTrunc *(PR [#3367](https://github.com/tobymao/sqlglot/pull/3367) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#3366](https://github.com/tobymao/sqlglot/issues/3366) opened by [@sivpr2000](https://github.com/sivpr2000)*
### :bug: Bug Fixes
- [`f697cb1`](https://github.com/tobymao/sqlglot/commit/f697cb16b6d744253febb2f83476853e63e06f88) - duckdb describe query closes [#3353](https://github.com/tobymao/sqlglot/pull/3353) *(commit by [@tobymao](https://github.com/tobymao))*
- [`6e0fc5d`](https://github.com/tobymao/sqlglot/commit/6e0fc5dd8e1921aac1e3f9834dd6a1c0e30b9e50) - export optimizer functions explicitly in init *(PR [#3358](https://github.com/tobymao/sqlglot/pull/3358) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3354](https://github.com/tobymao/sqlglot/issues/3354) opened by [@tekumara](https://github.com/tekumara)*
- [`23d45ee`](https://github.com/tobymao/sqlglot/commit/23d45eefb8b5f650d2e723499a12ac6801d5cd14) - **postgres**: don't generate CommentColumnConstraint *(PR [#3357](https://github.com/tobymao/sqlglot/pull/3357) by [@georgesittas](https://github.com/georgesittas))*
- [`e87685b`](https://github.com/tobymao/sqlglot/commit/e87685b6971d6ddb7d222993b38aa224c39c5154) - **lineage**: use source names of derived table sources for laterals *(PR [#3360](https://github.com/tobymao/sqlglot/pull/3360) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3356](https://github.com/tobymao/sqlglot/issues/3356) opened by [@eliaperantoni](https://github.com/eliaperantoni)*
- [`e82a30b`](https://github.com/tobymao/sqlglot/commit/e82a30b6563547daea0bb087e1b6b5bf3b0532d3) - **postgres**: don't generate SchemaCommentProperty *(PR [#3364](https://github.com/tobymao/sqlglot/pull/3364) by [@georgesittas](https://github.com/georgesittas))*
- [`47dc52c`](https://github.com/tobymao/sqlglot/commit/47dc52c99ea50b55d08f2b57885eebbd577b8b46) - **mysql**: convert epoch extraction into UNIX_TIMESTAMP call *(PR [#3369](https://github.com/tobymao/sqlglot/pull/3369) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3368](https://github.com/tobymao/sqlglot/issues/3368) opened by [@FaizelK](https://github.com/FaizelK)*
- [`b8f0979`](https://github.com/tobymao/sqlglot/commit/b8f0979537cf3ad9ef83f2c30d6cfb23cd4d2d1e) - **mysql**: generate GROUP_CONCAT for ArrayAgg *(PR [#3370](https://github.com/tobymao/sqlglot/pull/3370) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3368](https://github.com/tobymao/sqlglot/issues/3368) opened by [@FaizelK](https://github.com/FaizelK)*
### :recycle: Refactors
- [`b928f54`](https://github.com/tobymao/sqlglot/commit/b928f542a81d299311d01bd8f1eb762a13adf5c8) - don't mutate the AST when creating DDL scopes *(PR [#3371](https://github.com/tobymao/sqlglot/pull/3371) by [@georgesittas](https://github.com/georgesittas))*
## [v23.12.1] - 2024-04-25
### :wrench: Chores
- [`719d394`](https://github.com/tobymao/sqlglot/commit/719d3949b75bcdac0d19b86d7398c5d9c4b5bdc3) - add a test for quoted aliases *(commit by [@tobymao](https://github.com/tobymao))*
- [`6d7a9f4`](https://github.com/tobymao/sqlglot/commit/6d7a9f4ec0cd87efe19128dc9e55967172bf324e) - use unknown token types *(commit by [@tobymao](https://github.com/tobymao))*
## [v23.12.0] - 2024-04-25 ## [v23.12.0] - 2024-04-25
### :boom: BREAKING CHANGES ### :boom: BREAKING CHANGES
- due to [`c5ce47b`](https://github.com/tobymao/sqlglot/commit/c5ce47ba7863e0c536e076ea78ec27cb52324493) - Combine aggregate functions with orderby from WITHIN GROUP *(PR [#3352](https://github.com/tobymao/sqlglot/pull/3352) by [@VaggelisD](https://github.com/VaggelisD))*: - due to [`c5ce47b`](https://github.com/tobymao/sqlglot/commit/c5ce47ba7863e0c536e076ea78ec27cb52324493) - Combine aggregate functions with orderby from WITHIN GROUP *(PR [#3352](https://github.com/tobymao/sqlglot/pull/3352) by [@VaggelisD](https://github.com/VaggelisD))*:
@ -3358,3 +3438,6 @@ Changelog
[v23.11.1]: https://github.com/tobymao/sqlglot/compare/v23.11.0...v23.11.1 [v23.11.1]: https://github.com/tobymao/sqlglot/compare/v23.11.0...v23.11.1
[v23.11.2]: https://github.com/tobymao/sqlglot/compare/v23.11.1...v23.11.2 [v23.11.2]: https://github.com/tobymao/sqlglot/compare/v23.11.1...v23.11.2
[v23.12.0]: https://github.com/tobymao/sqlglot/compare/v23.11.2...v23.12.0 [v23.12.0]: https://github.com/tobymao/sqlglot/compare/v23.11.2...v23.12.0
[v23.12.1]: https://github.com/tobymao/sqlglot/compare/v23.12.0...v23.12.1
[v23.12.2]: https://github.com/tobymao/sqlglot/compare/v23.12.1...v23.12.2
[v23.13.0]: https://github.com/tobymao/sqlglot/compare/v23.12.2...v23.13.0

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

8
docs/sqlglot/_version.html
View file

@ -76,8 +76,8 @@
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> </span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;23.12.0&#39;</span> </span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;23.13.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">23</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> </span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">23</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -97,7 +97,7 @@
<section id="version"> <section id="version">
<div class="attr variable"> <div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> = <span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;23.12.0&#39;</span> <span class="default_value">&#39;23.13.0&#39;</span>
</div> </div>
@ -109,7 +109,7 @@
<section id="version_tuple"> <section id="version_tuple">
<div class="attr variable"> <div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> = <span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(23, 12, 0)</span> <span class="default_value">(23, 13, 0)</span>
</div> </div>

40
docs/sqlglot/dataframe/sql.html
View file

@ -791,7 +791,7 @@
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396128570384&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396128570384&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396128323856&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874099102576&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874099102576&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874118641712&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label> <label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
@ -1848,7 +1848,7 @@
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function"> <div class="attr function">
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396133005680&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o">&lt;</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396133947408&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span> <span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874103081520&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o">&lt;</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874102380192&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
@ -2096,7 +2096,7 @@
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396128968112&#39;</span><span class="o">&gt;</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span> <span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874099548832&#39;</span><span class="o">&gt;</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
<label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label>
@ -2846,7 +2846,7 @@ is unlikely to come up.</p>
<div class="decorator">@operation(Operation.FROM)</div> <div class="decorator">@operation(Operation.FROM)</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396129406016&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874098212304&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
@ -2915,7 +2915,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@operation(Operation.FROM)</div> <div class="decorator">@operation(Operation.FROM)</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396129194224&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396129194224&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874098399216&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874098399216&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
@ -3120,7 +3120,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@operation(Operation.NO_OP)</div> <div class="decorator">@operation(Operation.NO_OP)</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396129194224&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396129194224&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874098399216&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874098399216&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
@ -3840,7 +3840,7 @@ and check if it matches the type of the value provided. If not then make it null
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function"> <div class="attr function">
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span> <span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
@ -3884,7 +3884,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
@ -3905,7 +3905,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span> <span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span>
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
@ -3926,7 +3926,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
@ -3953,7 +3953,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
@ -3992,7 +3992,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
@ -4013,7 +4013,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
@ -4607,7 +4607,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">):</span></span> <span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
@ -4628,7 +4628,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396130510128&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874101586928&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
@ -4663,7 +4663,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396129954368&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097045296&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
@ -4908,7 +4908,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
@ -4929,7 +4929,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
@ -5171,7 +5171,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
@ -5198,7 +5198,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140396127480864&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139874097095024&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>

24
docs/sqlglot/dialects/athena.html
View file

File diff suppressed because one or more lines are too long

37
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

47
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

523
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

3828
docs/sqlglot/dialects/dialect.html
View file

File diff suppressed because it is too large Load diff

34
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

27
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

2266
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

33
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

1583
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

31
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

2220
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

2534
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

1020
docs/sqlglot/dialects/prql.html
View file

File diff suppressed because one or more lines are too long

2260
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

4219
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

507
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

195
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

29
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

34
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

25
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

35
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

355
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

2480
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

25
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

21428
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

19080
docs/sqlglot/generator.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/helper.html
View file

@ -1893,7 +1893,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS"> <section id="DATE_UNITS">
<div class="attr variable"> <div class="attr variable">
<span class="name">DATE_UNITS</span> = <span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;year_month&#39;, &#39;month&#39;, &#39;quarter&#39;, &#39;year&#39;, &#39;week&#39;, &#39;day&#39;}</span> <span class="default_value">{&#39;year&#39;, &#39;week&#39;, &#39;day&#39;, &#39;year_month&#39;, &#39;quarter&#39;, &#39;month&#39;}</span>
</div> </div>

2
docs/sqlglot/jsonpath.html
View file

@ -577,7 +577,7 @@
<div class="attr variable"> <div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> = <span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;}</span>
</div> </div>

1487
docs/sqlglot/lineage.html
View file

File diff suppressed because it is too large Load diff

14
docs/sqlglot/optimizer.html
View file

@ -73,14 +73,14 @@
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="c1"># ruff: noqa: F401</span> <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="c1"># ruff: noqa: F401</span>
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a> </span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.optimizer</span> <span class="kn">import</span> <span class="n">RULES</span><span class="p">,</span> <span class="n">optimize</span> </span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.optimizer</span> <span class="kn">import</span> <span class="n">RULES</span> <span class="k">as</span> <span class="n">RULES</span><span class="p">,</span> <span class="n">optimize</span> <span class="k">as</span> <span class="n">optimize</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="p">(</span> </span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="p">(</span>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a> <span class="n">Scope</span><span class="p">,</span> </span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a> <span class="n">Scope</span> <span class="k">as</span> <span class="n">Scope</span><span class="p">,</span>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a> <span class="n">build_scope</span><span class="p">,</span> </span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a> <span class="n">build_scope</span> <span class="k">as</span> <span class="n">build_scope</span><span class="p">,</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a> <span class="n">find_all_in_scope</span><span class="p">,</span> </span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a> <span class="n">find_all_in_scope</span> <span class="k">as</span> <span class="n">find_all_in_scope</span><span class="p">,</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a> <span class="n">find_in_scope</span><span class="p">,</span> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a> <span class="n">find_in_scope</span> <span class="k">as</span> <span class="n">find_in_scope</span><span class="p">,</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a> <span class="n">traverse_scope</span><span class="p">,</span> </span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a> <span class="n">traverse_scope</span> <span class="k">as</span> <span class="n">traverse_scope</span><span class="p">,</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a> <span class="n">walk_in_scope</span><span class="p">,</span> </span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a> <span class="n">walk_in_scope</span> <span class="k">as</span> <span class="n">walk_in_scope</span><span class="p">,</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="p">)</span> </span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="p">)</span>
</span></pre></div> </span></pre></div>

8
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

160
docs/sqlglot/optimizer/eliminate_subqueries.html
View file

@ -147,86 +147,87 @@
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">new_ctes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">new_cte</span><span class="p">)</span> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">new_ctes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">new_cte</span><span class="p">)</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> </span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="n">new_ctes</span><span class="p">:</span> </span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="n">new_ctes</span><span class="p">:</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="n">new_ctes</span><span class="p">,</span> <span class="n">recursive</span><span class="o">=</span><span class="n">recursive</span><span class="p">))</span> </span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DDL</span><span class="p">)</span> <span class="k">else</span> <span class="n">expression</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> </span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="n">new_ctes</span><span class="p">,</span> <span class="n">recursive</span><span class="o">=</span><span class="n">recursive</span><span class="p">))</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> </span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> </span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a><span class="k">def</span> <span class="nf">_eliminate</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span> </span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">is_derived_table</span><span class="p">:</span> </span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a><span class="k">def</span> <span class="nf">_eliminate</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">return</span> <span class="n">_eliminate_derived_table</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span> </span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">is_derived_table</span><span class="p">:</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> </span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">return</span> <span class="n">_eliminate_derived_table</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">is_cte</span><span class="p">:</span> </span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span> </span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">is_cte</span><span class="p">:</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> </span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">return</span> <span class="n">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> </span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a><span class="k">def</span> <span class="nf">_eliminate_derived_table</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span> </span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="c1"># This makes sure that we don&#39;t:</span> </span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="k">def</span> <span class="nf">_eliminate_derived_table</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="c1"># - drop the &quot;pivot&quot; arg from a pivoted subquery</span> </span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="c1"># This makes sure that we don&#39;t:</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># - eliminate a lateral correlated subquery</span> </span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># - drop the &quot;pivot&quot; arg from a pivoted subquery</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">pivots</span> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">):</span> </span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="c1"># - eliminate a lateral correlated subquery</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">return</span> <span class="kc">None</span> </span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">pivots</span> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">):</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> </span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="kc">None</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># Get rid of redundant exp.Subquery expressions, i.e. those that are just used as wrappers</span> </span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">unwrap</span><span class="p">()</span> </span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="c1"># Get rid of redundant exp.Subquery expressions, i.e. those that are just used as wrappers</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span> </span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">unwrap</span><span class="p">()</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">to_replace</span><span class="o">.</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span><span class="p">)</span> </span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">to_replace</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">))</span> </span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">to_replace</span><span class="o">.</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span><span class="p">)</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> </span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">to_replace</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">))</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> </span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> </span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">return</span> <span class="n">cte</span> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="n">cte</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="k">def</span> <span class="nf">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span> </span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a><span class="k">def</span> <span class="nf">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span> </span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> </span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">parent</span> </span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> </span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">parent</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span> </span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> </span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> </span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="c1"># Rename references to this CTE</span> </span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">traverse</span><span class="p">():</span> </span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="c1"># Rename references to this CTE</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">child_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span> </span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">traverse</span><span class="p">():</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">source</span> <span class="ow">is</span> <span class="n">scope</span><span class="p">:</span> </span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">child_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">new_table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">if</span> <span class="n">source</span> <span class="ow">is</span> <span class="n">scope</span><span class="p">:</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">table</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_table</span><span class="p">)</span> </span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">new_table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> </span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">table</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_table</span><span class="p">)</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">return</span> <span class="n">cte</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">return</span> <span class="n">cte</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a><span class="k">def</span> <span class="nf">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a><span class="k">def</span> <span class="nf">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a><span class="sd"> Returns:</span> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a><span class="sd"> tuple of (name, cte)</span> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a><span class="sd"> Returns:</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a><span class="sd"> where `name` is a new name for this CTE in the root scope and `cte` is a new CTE instance.</span> </span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a><span class="sd"> tuple of (name, cte)</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a><span class="sd"> If this CTE duplicates an existing CTE, `cte` will be None.</span> </span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a><span class="sd"> where `name` is a new name for this CTE in the root scope and `cte` is a new CTE instance.</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a><span class="sd"> If this CTE duplicates an existing CTE, `cte` will be None.</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">duplicate_cte_alias</span> <span class="o">=</span> <span class="n">existing_ctes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span> </span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">duplicate_cte_alias</span> <span class="o">=</span> <span class="n">existing_ctes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">alias</span> </span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> </span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="p">:</span> </span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s2">&quot;cte&quot;</span><span class="p">)</span> </span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="p">:</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> </span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s2">&quot;cte&quot;</span><span class="p">)</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span> </span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">duplicate_cte_alias</span> </span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">if</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">elif</span> <span class="n">taken</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">):</span> </span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">duplicate_cte_alias</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="n">name</span><span class="p">)</span> </span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">elif</span> <span class="n">taken</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> </span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="n">name</span><span class="p">)</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span> </span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> </span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span> </span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">existing_ctes</span><span class="p">[</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">]</span> <span class="o">=</span> <span class="n">name</span> </span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">(</span> </span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">existing_ctes</span><span class="p">[</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">]</span> <span class="o">=</span> <span class="n">name</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">this</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> </span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">(</span>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span> </span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="n">this</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="p">)</span> </span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="p">)</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="kc">None</span> </span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">return</span> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> </span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="kc">None</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">return</span> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span>
</span></pre></div> </span></pre></div>
@ -326,9 +327,10 @@
</span><span id="eliminate_subqueries-90"><a href="#eliminate_subqueries-90"><span class="linenos">90</span></a> <span class="n">new_ctes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">new_cte</span><span class="p">)</span> </span><span id="eliminate_subqueries-90"><a href="#eliminate_subqueries-90"><span class="linenos">90</span></a> <span class="n">new_ctes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">new_cte</span><span class="p">)</span>
</span><span id="eliminate_subqueries-91"><a href="#eliminate_subqueries-91"><span class="linenos">91</span></a> </span><span id="eliminate_subqueries-91"><a href="#eliminate_subqueries-91"><span class="linenos">91</span></a>
</span><span id="eliminate_subqueries-92"><a href="#eliminate_subqueries-92"><span class="linenos">92</span></a> <span class="k">if</span> <span class="n">new_ctes</span><span class="p">:</span> </span><span id="eliminate_subqueries-92"><a href="#eliminate_subqueries-92"><span class="linenos">92</span></a> <span class="k">if</span> <span class="n">new_ctes</span><span class="p">:</span>
</span><span id="eliminate_subqueries-93"><a href="#eliminate_subqueries-93"><span class="linenos">93</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="n">new_ctes</span><span class="p">,</span> <span class="n">recursive</span><span class="o">=</span><span class="n">recursive</span><span class="p">))</span> </span><span id="eliminate_subqueries-93"><a href="#eliminate_subqueries-93"><span class="linenos">93</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DDL</span><span class="p">)</span> <span class="k">else</span> <span class="n">expression</span>
</span><span id="eliminate_subqueries-94"><a href="#eliminate_subqueries-94"><span class="linenos">94</span></a> </span><span id="eliminate_subqueries-94"><a href="#eliminate_subqueries-94"><span class="linenos">94</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="n">new_ctes</span><span class="p">,</span> <span class="n">recursive</span><span class="o">=</span><span class="n">recursive</span><span class="p">))</span>
</span><span id="eliminate_subqueries-95"><a href="#eliminate_subqueries-95"><span class="linenos">95</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="eliminate_subqueries-95"><a href="#eliminate_subqueries-95"><span class="linenos">95</span></a>
</span><span id="eliminate_subqueries-96"><a href="#eliminate_subqueries-96"><span class="linenos">96</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>

2
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -586,7 +586,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> = <span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;limit&#39;, &#39;kind&#39;, &#39;locks&#39;, &#39;distribute&#39;, &#39;settings&#39;, &#39;with&#39;, &#39;format&#39;, &#39;sample&#39;, &#39;cluster&#39;, &#39;group&#39;, &#39;having&#39;, &#39;windows&#39;, &#39;distinct&#39;, &#39;match&#39;, &#39;prewhere&#39;, &#39;connect&#39;, &#39;qualify&#39;, &#39;into&#39;, &#39;offset&#39;, &#39;laterals&#39;, &#39;pivots&#39;, &#39;options&#39;, &#39;sort&#39;}</span> <label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;offset&#39;, &#39;qualify&#39;, &#39;having&#39;, &#39;pivots&#39;, &#39;connect&#39;, &#39;sort&#39;, &#39;limit&#39;, &#39;windows&#39;, &#39;match&#39;, &#39;settings&#39;, &#39;group&#39;, &#39;options&#39;, &#39;format&#39;, &#39;sample&#39;, &#39;with&#39;, &#39;into&#39;, &#39;prewhere&#39;, &#39;distribute&#39;, &#39;distinct&#39;, &#39;cluster&#39;, &#39;locks&#39;, &#39;laterals&#39;, &#39;kind&#39;}</span>
</div> </div>

3445
docs/sqlglot/optimizer/scope.html
View file

File diff suppressed because it is too large Load diff

4184
docs/sqlglot/optimizer/simplify.html
View file

File diff suppressed because it is too large Load diff

23932
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

4923
docs/sqlglot/tokens.html
View file

File diff suppressed because one or more lines are too long

3
sqlglot/dialects/databricks.py
View file

@ -18,6 +18,7 @@ def _timestamp_diff(
class Databricks(Spark): class Databricks(Spark):
SAFE_DIVISION = False SAFE_DIVISION = False
COPY_PARAMS_ARE_CSV = False
class Parser(Spark.Parser): class Parser(Spark.Parser):
LOG_DEFAULTS_TO_LN = True LOG_DEFAULTS_TO_LN = True
@ -38,6 +39,8 @@ class Databricks(Spark):
class Generator(Spark.Generator): class Generator(Spark.Generator):
TABLESAMPLE_SEED_KEYWORD = "REPEATABLE" TABLESAMPLE_SEED_KEYWORD = "REPEATABLE"
COPY_PARAMS_ARE_WRAPPED = False
COPY_PARAMS_EQ_REQUIRED = True
TRANSFORMS = { TRANSFORMS = {
**Spark.Generator.TRANSFORMS, **Spark.Generator.TRANSFORMS,

10
sqlglot/dialects/dialect.py
View file

@ -161,6 +161,9 @@ class _Dialect(type):
if enum not in ("", "bigquery"): if enum not in ("", "bigquery"):
klass.generator_class.SELECT_KINDS = () klass.generator_class.SELECT_KINDS = ()
if enum not in ("", "athena", "presto", "trino"):
klass.generator_class.TRY_SUPPORTED = False
if enum not in ("", "databricks", "hive", "spark", "spark2"): if enum not in ("", "databricks", "hive", "spark", "spark2"):
modifier_transforms = klass.generator_class.AFTER_HAVING_MODIFIER_TRANSFORMS.copy() modifier_transforms = klass.generator_class.AFTER_HAVING_MODIFIER_TRANSFORMS.copy()
for modifier in ("cluster", "distribute", "sort"): for modifier in ("cluster", "distribute", "sort"):
@ -318,6 +321,9 @@ class Dialect(metaclass=_Dialect):
UNICODE_START: t.Optional[str] = None UNICODE_START: t.Optional[str] = None
UNICODE_END: t.Optional[str] = None UNICODE_END: t.Optional[str] = None
# Separator of COPY statement parameters
COPY_PARAMS_ARE_CSV = True
@classmethod @classmethod
def get_or_raise(cls, dialect: DialectType) -> Dialect: def get_or_raise(cls, dialect: DialectType) -> Dialect:
""" """
@ -897,9 +903,7 @@ def regexp_extract_sql(self: Generator, expression: exp.RegexpExtract) -> str:
def regexp_replace_sql(self: Generator, expression: exp.RegexpReplace) -> str: def regexp_replace_sql(self: Generator, expression: exp.RegexpReplace) -> str:
bad_args = list( bad_args = list(filter(expression.args.get, ("position", "occurrence", "modifiers")))
filter(expression.args.get, ("position", "occurrence", "parameters", "modifiers"))
)
if bad_args: if bad_args:
self.unsupported(f"REGEXP_REPLACE does not support the following arg(s): {bad_args}") self.unsupported(f"REGEXP_REPLACE does not support the following arg(s): {bad_args}")

3
sqlglot/dialects/duckdb.py
View file

@ -280,6 +280,8 @@ class DuckDB(Dialect):
"RANGE": _build_generate_series(end_exclusive=True), "RANGE": _build_generate_series(end_exclusive=True),
} }
FUNCTIONS.pop("DATE_SUB")
FUNCTION_PARSERS = parser.Parser.FUNCTION_PARSERS.copy() FUNCTION_PARSERS = parser.Parser.FUNCTION_PARSERS.copy()
FUNCTION_PARSERS.pop("DECODE") FUNCTION_PARSERS.pop("DECODE")
@ -365,6 +367,7 @@ class DuckDB(Dialect):
MULTI_ARG_DISTINCT = False MULTI_ARG_DISTINCT = False
CAN_IMPLEMENT_ARRAY_ANY = True CAN_IMPLEMENT_ARRAY_ANY = True
SUPPORTS_TO_NUMBER = False SUPPORTS_TO_NUMBER = False
COPY_HAS_INTO_KEYWORD = False
TRANSFORMS = { TRANSFORMS = {
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,

53
sqlglot/dialects/mysql.py
View file

@ -668,6 +668,7 @@ class MySQL(Dialect):
TRANSFORMS = { TRANSFORMS = {
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,
exp.ArrayAgg: rename_func("GROUP_CONCAT"),
exp.CurrentDate: no_paren_current_date_sql, exp.CurrentDate: no_paren_current_date_sql,
exp.DateDiff: _remove_ts_or_ds_to_date( exp.DateDiff: _remove_ts_or_ds_to_date(
lambda self, e: self.func("DATEDIFF", e.this, e.expression), ("this", "expression") lambda self, e: self.func("DATEDIFF", e.this, e.expression), ("this", "expression")
@ -766,15 +767,37 @@ class MySQL(Dialect):
LIMIT_ONLY_LITERALS = True LIMIT_ONLY_LITERALS = True
CHAR_CAST_MAPPING = dict.fromkeys(
(
exp.DataType.Type.LONGTEXT,
exp.DataType.Type.LONGBLOB,
exp.DataType.Type.MEDIUMBLOB,
exp.DataType.Type.MEDIUMTEXT,
exp.DataType.Type.TEXT,
exp.DataType.Type.TINYBLOB,
exp.DataType.Type.TINYTEXT,
exp.DataType.Type.VARCHAR,
),
"CHAR",
)
SIGNED_CAST_MAPPING = dict.fromkeys(
(
exp.DataType.Type.BIGINT,
exp.DataType.Type.BOOLEAN,
exp.DataType.Type.INT,
exp.DataType.Type.SMALLINT,
exp.DataType.Type.TINYINT,
exp.DataType.Type.MEDIUMINT,
),
"SIGNED",
)
# MySQL doesn't support many datatypes in cast. # MySQL doesn't support many datatypes in cast.
# https://dev.mysql.com/doc/refman/8.0/en/cast-functions.html#function_cast # https://dev.mysql.com/doc/refman/8.0/en/cast-functions.html#function_cast
CAST_MAPPING = { CAST_MAPPING = {
exp.DataType.Type.BIGINT: "SIGNED", **CHAR_CAST_MAPPING,
exp.DataType.Type.BOOLEAN: "SIGNED", **SIGNED_CAST_MAPPING,
exp.DataType.Type.INT: "SIGNED",
exp.DataType.Type.TEXT: "CHAR",
exp.DataType.Type.UBIGINT: "UNSIGNED", exp.DataType.Type.UBIGINT: "UNSIGNED",
exp.DataType.Type.VARCHAR: "CHAR",
} }
TIMESTAMP_FUNC_TYPES = { TIMESTAMP_FUNC_TYPES = {
@ -782,6 +805,13 @@ class MySQL(Dialect):
exp.DataType.Type.TIMESTAMPLTZ, exp.DataType.Type.TIMESTAMPLTZ,
} }
def extract_sql(self, expression: exp.Extract) -> str:
unit = expression.name
if unit and unit.lower() == "epoch":
return self.func("UNIX_TIMESTAMP", expression.expression)
return super().extract_sql(expression)
def datatype_sql(self, expression: exp.DataType) -> str: def datatype_sql(self, expression: exp.DataType) -> str:
# https://dev.mysql.com/doc/refman/8.0/en/numeric-type-syntax.html # https://dev.mysql.com/doc/refman/8.0/en/numeric-type-syntax.html
result = super().datatype_sql(expression) result = super().datatype_sql(expression)
@ -867,3 +897,16 @@ class MySQL(Dialect):
charset = expression.args.get("charset") charset = expression.args.get("charset")
using = f" USING {self.sql(charset)}" if charset else "" using = f" USING {self.sql(charset)}" if charset else ""
return f"CHAR({this}{using})" return f"CHAR({this}{using})"
def timestamptrunc_sql(self, expression: exp.TimestampTrunc) -> str:
unit = expression.args.get("unit")
# Pick an old-enough date to avoid negative timestamp diffs
start_ts = "'0000-01-01 00:00:00'"
# Source: https://stackoverflow.com/a/32955740
timestamp_diff = build_date_delta(exp.TimestampDiff)([unit, start_ts, expression.this])
interval = exp.Interval(this=timestamp_diff, unit=unit)
dateadd = build_date_delta_with_interval(exp.DateAdd)([start_ts, interval])
return self.sql(dateadd)

47
sqlglot/dialects/postgres.py
View file

@ -32,7 +32,7 @@ from sqlglot.dialects.dialect import (
trim_sql, trim_sql,
ts_or_ds_add_cast, ts_or_ds_add_cast,
) )
from sqlglot.helper import seq_get from sqlglot.helper import is_int, seq_get
from sqlglot.parser import binary_range_parser from sqlglot.parser import binary_range_parser
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
@ -204,6 +204,29 @@ def _json_extract_sql(
return _generate return _generate
def _build_regexp_replace(args: t.List) -> exp.RegexpReplace:
# The signature of REGEXP_REPLACE is:
# regexp_replace(source, pattern, replacement [, start [, N ]] [, flags ])
#
# Any one of `start`, `N` and `flags` can be column references, meaning that
# unless we can statically see that the last argument is a non-integer string
# (eg. not '0'), then it's not possible to construct the correct AST
if len(args) > 3:
last = args[-1]
if not is_int(last.name):
if not last.type or last.is_type(exp.DataType.Type.UNKNOWN, exp.DataType.Type.NULL):
from sqlglot.optimizer.annotate_types import annotate_types
last = annotate_types(last)
if last.is_type(*exp.DataType.TEXT_TYPES):
regexp_replace = exp.RegexpReplace.from_arg_list(args[:-1])
regexp_replace.set("modifiers", last)
return regexp_replace
return exp.RegexpReplace.from_arg_list(args)
class Postgres(Dialect): class Postgres(Dialect):
INDEX_OFFSET = 1 INDEX_OFFSET = 1
TYPED_DIVISION = True TYPED_DIVISION = True
@ -266,24 +289,25 @@ class Postgres(Dialect):
"BIGSERIAL": TokenType.BIGSERIAL, "BIGSERIAL": TokenType.BIGSERIAL,
"CHARACTER VARYING": TokenType.VARCHAR, "CHARACTER VARYING": TokenType.VARCHAR,
"CONSTRAINT TRIGGER": TokenType.COMMAND, "CONSTRAINT TRIGGER": TokenType.COMMAND,
"CSTRING": TokenType.PSEUDO_TYPE,
"DECLARE": TokenType.COMMAND, "DECLARE": TokenType.COMMAND,
"DO": TokenType.COMMAND, "DO": TokenType.COMMAND,
"EXEC": TokenType.COMMAND, "EXEC": TokenType.COMMAND,
"HSTORE": TokenType.HSTORE, "HSTORE": TokenType.HSTORE,
"INT8": TokenType.BIGINT,
"JSONB": TokenType.JSONB, "JSONB": TokenType.JSONB,
"MONEY": TokenType.MONEY, "MONEY": TokenType.MONEY,
"NAME": TokenType.NAME,
"OID": TokenType.OBJECT_IDENTIFIER,
"ONLY": TokenType.ONLY,
"OPERATOR": TokenType.OPERATOR,
"REFRESH": TokenType.COMMAND, "REFRESH": TokenType.COMMAND,
"REINDEX": TokenType.COMMAND, "REINDEX": TokenType.COMMAND,
"RESET": TokenType.COMMAND, "RESET": TokenType.COMMAND,
"REVOKE": TokenType.COMMAND, "REVOKE": TokenType.COMMAND,
"SERIAL": TokenType.SERIAL, "SERIAL": TokenType.SERIAL,
"SMALLSERIAL": TokenType.SMALLSERIAL, "SMALLSERIAL": TokenType.SMALLSERIAL,
"NAME": TokenType.NAME,
"TEMP": TokenType.TEMPORARY, "TEMP": TokenType.TEMPORARY,
"CSTRING": TokenType.PSEUDO_TYPE,
"OID": TokenType.OBJECT_IDENTIFIER,
"ONLY": TokenType.ONLY,
"OPERATOR": TokenType.OPERATOR,
"REGCLASS": TokenType.OBJECT_IDENTIFIER, "REGCLASS": TokenType.OBJECT_IDENTIFIER,
"REGCOLLATION": TokenType.OBJECT_IDENTIFIER, "REGCOLLATION": TokenType.OBJECT_IDENTIFIER,
"REGCONFIG": TokenType.OBJECT_IDENTIFIER, "REGCONFIG": TokenType.OBJECT_IDENTIFIER,
@ -320,6 +344,7 @@ class Postgres(Dialect):
"MAKE_TIME": exp.TimeFromParts.from_arg_list, "MAKE_TIME": exp.TimeFromParts.from_arg_list,
"MAKE_TIMESTAMP": exp.TimestampFromParts.from_arg_list, "MAKE_TIMESTAMP": exp.TimestampFromParts.from_arg_list,
"NOW": exp.CurrentTimestamp.from_arg_list, "NOW": exp.CurrentTimestamp.from_arg_list,
"REGEXP_REPLACE": _build_regexp_replace,
"TO_CHAR": build_formatted_time(exp.TimeToStr, "postgres"), "TO_CHAR": build_formatted_time(exp.TimeToStr, "postgres"),
"TO_TIMESTAMP": _build_to_timestamp, "TO_TIMESTAMP": _build_to_timestamp,
"UNNEST": exp.Explode.from_arg_list, "UNNEST": exp.Explode.from_arg_list,
@ -417,6 +442,7 @@ class Postgres(Dialect):
LIKE_PROPERTY_INSIDE_SCHEMA = True LIKE_PROPERTY_INSIDE_SCHEMA = True
MULTI_ARG_DISTINCT = False MULTI_ARG_DISTINCT = False
CAN_IMPLEMENT_ARRAY_ANY = True CAN_IMPLEMENT_ARRAY_ANY = True
COPY_HAS_INTO_KEYWORD = False
SUPPORTED_JSON_PATH_PARTS = { SUPPORTED_JSON_PATH_PARTS = {
exp.JSONPathKey, exp.JSONPathKey,
@ -518,6 +544,7 @@ class Postgres(Dialect):
exp.Variance: rename_func("VAR_SAMP"), exp.Variance: rename_func("VAR_SAMP"),
exp.Xor: bool_xor_sql, exp.Xor: bool_xor_sql,
} }
TRANSFORMS.pop(exp.CommentColumnConstraint)
PROPERTIES_LOCATION = { PROPERTIES_LOCATION = {
**generator.Generator.PROPERTIES_LOCATION, **generator.Generator.PROPERTIES_LOCATION,
@ -526,6 +553,14 @@ class Postgres(Dialect):
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED, exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
} }
def schemacommentproperty_sql(self, expression: exp.SchemaCommentProperty) -> str:
self.unsupported("Table comments are not supported in the CREATE statement")
return ""
def commentcolumnconstraint_sql(self, expression: exp.CommentColumnConstraint) -> str:
self.unsupported("Column comments are not supported in the CREATE statement")
return ""
def unnest_sql(self, expression: exp.Unnest) -> str: def unnest_sql(self, expression: exp.Unnest) -> str:
if len(expression.expressions) == 1: if len(expression.expressions) == 1:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types

3
sqlglot/dialects/presto.py
View file

@ -222,6 +222,8 @@ class Presto(Dialect):
"ROW": TokenType.STRUCT, "ROW": TokenType.STRUCT,
"IPADDRESS": TokenType.IPADDRESS, "IPADDRESS": TokenType.IPADDRESS,
"IPPREFIX": TokenType.IPPREFIX, "IPPREFIX": TokenType.IPPREFIX,
"TDIGEST": TokenType.TDIGEST,
"HYPERLOGLOG": TokenType.HLLSKETCH,
} }
KEYWORDS.pop("QUALIFY") KEYWORDS.pop("QUALIFY")
@ -316,6 +318,7 @@ class Presto(Dialect):
exp.DataType.Type.STRUCT: "ROW", exp.DataType.Type.STRUCT: "ROW",
exp.DataType.Type.DATETIME: "TIMESTAMP", exp.DataType.Type.DATETIME: "TIMESTAMP",
exp.DataType.Type.DATETIME64: "TIMESTAMP", exp.DataType.Type.DATETIME64: "TIMESTAMP",
exp.DataType.Type.HLLSKETCH: "HYPERLOGLOG",
} }
TRANSFORMS = { TRANSFORMS = {

40
sqlglot/dialects/prql.py
View file

@ -4,6 +4,7 @@ import typing as t
from sqlglot import exp, parser, tokens from sqlglot import exp, parser, tokens
from sqlglot.dialects.dialect import Dialect from sqlglot.dialects.dialect import Dialect
from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
@ -53,6 +54,15 @@ class PRQL(Dialect):
_select_all(self._parse_table()), distinct=False, copy=False _select_all(self._parse_table()), distinct=False, copy=False
), ),
"SORT": lambda self, query: self._parse_order_by(query), "SORT": lambda self, query: self._parse_order_by(query),
"AGGREGATE": lambda self, query: self._parse_selection(
query, parse_method=self._parse_aggregate, append=False
),
}
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"AVERAGE": exp.Avg.from_arg_list,
"SUM": lambda args: exp.func("COALESCE", exp.Sum(this=seq_get(args, 0)), 0),
} }
def _parse_equality(self) -> t.Optional[exp.Expression]: def _parse_equality(self) -> t.Optional[exp.Expression]:
@ -87,14 +97,20 @@ class PRQL(Dialect):
return query return query
def _parse_selection(self, query: exp.Query, append: bool = True) -> exp.Query: def _parse_selection(
self,
query: exp.Query,
parse_method: t.Optional[t.Callable] = None,
append: bool = True,
) -> exp.Query:
parse_method = parse_method if parse_method else self._parse_expression
if self._match(TokenType.L_BRACE): if self._match(TokenType.L_BRACE):
selects = self._parse_csv(self._parse_expression) selects = self._parse_csv(parse_method)
if not self._match(TokenType.R_BRACE, expression=query): if not self._match(TokenType.R_BRACE, expression=query):
self.raise_error("Expecting }") self.raise_error("Expecting }")
else: else:
expression = self._parse_expression() expression = parse_method()
selects = [expression] if expression else [] selects = [expression] if expression else []
projections = { projections = {
@ -136,6 +152,24 @@ class PRQL(Dialect):
self.raise_error("Expecting }") self.raise_error("Expecting }")
return query.order_by(self.expression(exp.Order, expressions=expressions), copy=False) return query.order_by(self.expression(exp.Order, expressions=expressions), copy=False)
def _parse_aggregate(self) -> t.Optional[exp.Expression]:
alias = None
if self._next and self._next.token_type == TokenType.ALIAS:
alias = self._parse_id_var(any_token=True)
self._match(TokenType.ALIAS)
name = self._curr and self._curr.text.upper()
func_builder = self.FUNCTIONS.get(name)
if func_builder:
self._advance()
args = self._parse_column()
func = func_builder([args])
else:
self.raise_error(f"Unsupported aggregation function {name}")
if alias:
return self.expression(exp.Alias, this=func, alias=alias)
return func
def _parse_expression(self) -> t.Optional[exp.Expression]: def _parse_expression(self) -> t.Optional[exp.Expression]:
if self._next and self._next.token_type == TokenType.ALIAS: if self._next and self._next.token_type == TokenType.ALIAS:
alias = self._parse_id_var(True) alias = self._parse_id_var(True)

2
sqlglot/dialects/redshift.py
View file

@ -38,6 +38,7 @@ class Redshift(Postgres):
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
INDEX_OFFSET = 0 INDEX_OFFSET = 0
COPY_PARAMS_ARE_CSV = False
TIME_FORMAT = "'YYYY-MM-DD HH:MI:SS'" TIME_FORMAT = "'YYYY-MM-DD HH:MI:SS'"
TIME_MAPPING = { TIME_MAPPING = {
@ -138,6 +139,7 @@ class Redshift(Postgres):
LAST_DAY_SUPPORTS_DATE_PART = False LAST_DAY_SUPPORTS_DATE_PART = False
CAN_IMPLEMENT_ARRAY_ANY = False CAN_IMPLEMENT_ARRAY_ANY = False
MULTI_ARG_DISTINCT = True MULTI_ARG_DISTINCT = True
COPY_PARAMS_ARE_WRAPPED = False
TYPE_MAPPING = { TYPE_MAPPING = {
**Postgres.Generator.TYPE_MAPPING, **Postgres.Generator.TYPE_MAPPING,

22
sqlglot/dialects/snowflake.py
View file

@ -289,6 +289,7 @@ class Snowflake(Dialect):
SUPPORTS_SEMI_ANTI_JOIN = False SUPPORTS_SEMI_ANTI_JOIN = False
PREFER_CTE_ALIAS_COLUMN = True PREFER_CTE_ALIAS_COLUMN = True
TABLESAMPLE_SIZE_IS_PERCENT = True TABLESAMPLE_SIZE_IS_PERCENT = True
COPY_PARAMS_ARE_CSV = False
TIME_MAPPING = { TIME_MAPPING = {
"YYYY": "%Y", "YYYY": "%Y",
@ -439,7 +440,7 @@ class Snowflake(Dialect):
PROPERTY_PARSERS = { PROPERTY_PARSERS = {
**parser.Parser.PROPERTY_PARSERS, **parser.Parser.PROPERTY_PARSERS,
"LOCATION": lambda self: self._parse_location(), "LOCATION": lambda self: self._parse_location_property(),
} }
SHOW_PARSERS = { SHOW_PARSERS = {
@ -675,10 +676,13 @@ class Snowflake(Dialect):
self._match_text_seq("WITH") self._match_text_seq("WITH")
return self.expression(exp.SwapTable, this=self._parse_table(schema=True)) return self.expression(exp.SwapTable, this=self._parse_table(schema=True))
def _parse_location(self) -> exp.LocationProperty: def _parse_location_property(self) -> exp.LocationProperty:
self._match(TokenType.EQ) self._match(TokenType.EQ)
return self.expression(exp.LocationProperty, this=self._parse_location_path()) return self.expression(exp.LocationProperty, this=self._parse_location_path())
def _parse_file_location(self) -> t.Optional[exp.Expression]:
return self._parse_table_parts()
def _parse_location_path(self) -> exp.Var: def _parse_location_path(self) -> exp.Var:
parts = [self._advance_any(ignore_reserved=True)] parts = [self._advance_any(ignore_reserved=True)]
@ -715,10 +719,7 @@ class Snowflake(Dialect):
"SQL_DOUBLE": TokenType.DOUBLE, "SQL_DOUBLE": TokenType.DOUBLE,
"SQL_VARCHAR": TokenType.VARCHAR, "SQL_VARCHAR": TokenType.VARCHAR,
"STORAGE INTEGRATION": TokenType.STORAGE_INTEGRATION, "STORAGE INTEGRATION": TokenType.STORAGE_INTEGRATION,
"TIMESTAMP_LTZ": TokenType.TIMESTAMPLTZ,
"TIMESTAMP_NTZ": TokenType.TIMESTAMP,
"TIMESTAMP_TZ": TokenType.TIMESTAMPTZ, "TIMESTAMP_TZ": TokenType.TIMESTAMPTZ,
"TIMESTAMPNTZ": TokenType.TIMESTAMP,
"TOP": TokenType.TOP, "TOP": TokenType.TOP,
} }
@ -745,6 +746,8 @@ class Snowflake(Dialect):
JSON_KEY_VALUE_PAIR_SEP = "," JSON_KEY_VALUE_PAIR_SEP = ","
INSERT_OVERWRITE = " OVERWRITE INTO" INSERT_OVERWRITE = " OVERWRITE INTO"
STRUCT_DELIMITER = ("(", ")") STRUCT_DELIMITER = ("(", ")")
COPY_PARAMS_ARE_WRAPPED = False
COPY_PARAMS_EQ_REQUIRED = True
TRANSFORMS = { TRANSFORMS = {
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,
@ -845,7 +848,6 @@ class Snowflake(Dialect):
**generator.Generator.TYPE_MAPPING, **generator.Generator.TYPE_MAPPING,
exp.DataType.Type.NESTED: "OBJECT", exp.DataType.Type.NESTED: "OBJECT",
exp.DataType.Type.STRUCT: "OBJECT", exp.DataType.Type.STRUCT: "OBJECT",
exp.DataType.Type.TIMESTAMP: "TIMESTAMPNTZ",
} }
STAR_MAPPING = { STAR_MAPPING = {
@ -1038,3 +1040,11 @@ class Snowflake(Dialect):
values.append(e) values.append(e)
return self.func("OBJECT_CONSTRUCT", *flatten(zip(keys, values))) return self.func("OBJECT_CONSTRUCT", *flatten(zip(keys, values)))
def copyparameter_sql(self, expression: exp.CopyParameter) -> str:
option = self.sql(expression, "this")
if option.upper() == "FILE_FORMAT":
values = self.expressions(expression, key="expression", flat=True, sep=" ")
return f"{option} = ({values})"
return super().copyparameter_sql(expression)

6
sqlglot/dialects/spark.py
View file

@ -5,7 +5,7 @@ import typing as t
from sqlglot import exp from sqlglot import exp
from sqlglot.dialects.dialect import rename_func, unit_to_var from sqlglot.dialects.dialect import rename_func, unit_to_var
from sqlglot.dialects.hive import _build_with_ignore_nulls from sqlglot.dialects.hive import _build_with_ignore_nulls
from sqlglot.dialects.spark2 import Spark2, temporary_storage_provider from sqlglot.dialects.spark2 import Spark2, temporary_storage_provider, _build_as_cast
from sqlglot.helper import ensure_list, seq_get from sqlglot.helper import ensure_list, seq_get
from sqlglot.transforms import ( from sqlglot.transforms import (
ctas_with_tmp_tables_to_create_tmp_view, ctas_with_tmp_tables_to_create_tmp_view,
@ -63,6 +63,8 @@ class Spark(Spark2):
**Spark2.Parser.FUNCTIONS, **Spark2.Parser.FUNCTIONS,
"ANY_VALUE": _build_with_ignore_nulls(exp.AnyValue), "ANY_VALUE": _build_with_ignore_nulls(exp.AnyValue),
"DATEDIFF": _build_datediff, "DATEDIFF": _build_datediff,
"TIMESTAMP_LTZ": _build_as_cast("TIMESTAMP_LTZ"),
"TIMESTAMP_NTZ": _build_as_cast("TIMESTAMP_NTZ"),
"TRY_ELEMENT_AT": lambda args: exp.Bracket( "TRY_ELEMENT_AT": lambda args: exp.Bracket(
this=seq_get(args, 0), expressions=ensure_list(seq_get(args, 1)), safe=True this=seq_get(args, 0), expressions=ensure_list(seq_get(args, 1)), safe=True
), ),
@ -88,6 +90,8 @@ class Spark(Spark2):
exp.DataType.Type.MONEY: "DECIMAL(15, 4)", exp.DataType.Type.MONEY: "DECIMAL(15, 4)",
exp.DataType.Type.SMALLMONEY: "DECIMAL(6, 4)", exp.DataType.Type.SMALLMONEY: "DECIMAL(6, 4)",
exp.DataType.Type.UNIQUEIDENTIFIER: "STRING", exp.DataType.Type.UNIQUEIDENTIFIER: "STRING",
exp.DataType.Type.TIMESTAMPLTZ: "TIMESTAMP_LTZ",
exp.DataType.Type.TIMESTAMPNTZ: "TIMESTAMP_NTZ",
} }
TRANSFORMS = { TRANSFORMS = {

9
sqlglot/dialects/spark2.py
View file

@ -259,12 +259,15 @@ class Spark2(Hive):
return Generator.struct_sql(self, expression) return Generator.struct_sql(self, expression)
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str: def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
if is_parse_json(expression.this): arg = expression.this
is_json_extract = isinstance(arg, (exp.JSONExtract, exp.JSONExtractScalar))
if is_parse_json(arg) or is_json_extract:
schema = f"'{self.sql(expression, 'to')}'" schema = f"'{self.sql(expression, 'to')}'"
return self.func("FROM_JSON", expression.this.this, schema) return self.func("FROM_JSON", arg if is_json_extract else arg.this, schema)
if is_parse_json(expression): if is_parse_json(expression):
return self.func("TO_JSON", expression.this) return self.func("TO_JSON", arg)
return super(Hive.Generator, self).cast_sql(expression, safe_prefix=safe_prefix) return super(Hive.Generator, self).cast_sql(expression, safe_prefix=safe_prefix)

9
sqlglot/dialects/trino.py
View file

@ -1,7 +1,7 @@
from __future__ import annotations from __future__ import annotations
from sqlglot import exp from sqlglot import exp
from sqlglot.dialects.dialect import merge_without_target_sql from sqlglot.dialects.dialect import merge_without_target_sql, trim_sql
from sqlglot.dialects.presto import Presto from sqlglot.dialects.presto import Presto
@ -9,12 +9,19 @@ class Trino(Presto):
SUPPORTS_USER_DEFINED_TYPES = False SUPPORTS_USER_DEFINED_TYPES = False
LOG_BASE_FIRST = True LOG_BASE_FIRST = True
class Parser(Presto.Parser):
FUNCTION_PARSERS = {
**Presto.Parser.FUNCTION_PARSERS,
"TRIM": lambda self: self._parse_trim(),
}
class Generator(Presto.Generator): class Generator(Presto.Generator):
TRANSFORMS = { TRANSFORMS = {
**Presto.Generator.TRANSFORMS, **Presto.Generator.TRANSFORMS,
exp.ArraySum: lambda self, exp.ArraySum: lambda self,
e: f"REDUCE({self.sql(e, 'this')}, 0, (acc, x) -> acc + x, acc -> acc)", e: f"REDUCE({self.sql(e, 'this')}, 0, (acc, x) -> acc + x, acc -> acc)",
exp.Merge: merge_without_target_sql, exp.Merge: merge_without_target_sql,
exp.Trim: trim_sql,
} }
SUPPORTED_JSON_PATH_PARTS = { SUPPORTED_JSON_PATH_PARTS = {

3
sqlglot/dialects/tsql.py
View file

@ -728,6 +728,7 @@ class TSQL(Dialect):
JSON_PATH_BRACKETED_KEY_SUPPORTED = False JSON_PATH_BRACKETED_KEY_SUPPORTED = False
SUPPORTS_TO_NUMBER = False SUPPORTS_TO_NUMBER = False
OUTER_UNION_MODIFIERS = False OUTER_UNION_MODIFIERS = False
COPY_PARAMS_EQ_REQUIRED = True
EXPRESSIONS_WITHOUT_NESTED_CTES = { EXPRESSIONS_WITHOUT_NESTED_CTES = {
exp.Delete, exp.Delete,
@ -912,7 +913,7 @@ class TSQL(Dialect):
isinstance(prop, exp.TemporaryProperty) isinstance(prop, exp.TemporaryProperty)
for prop in (properties.expressions if properties else []) for prop in (properties.expressions if properties else [])
): ):
sql = f"#{sql}" sql = f"[#{sql[1:]}" if sql.startswith("[") else f"#{sql}"
return sql return sql

62
sqlglot/expressions.py
View file

@ -1955,6 +1955,31 @@ class Connect(Expression):
arg_types = {"start": False, "connect": True, "nocycle": False} arg_types = {"start": False, "connect": True, "nocycle": False}
class CopyParameter(Expression):
arg_types = {"this": True, "expression": False}
class Copy(Expression):
arg_types = {
"this": True,
"kind": True,
"files": True,
"credentials": False,
"format": False,
"params": False,
}
class Credentials(Expression):
arg_types = {
"credentials": False,
"encryption": False,
"storage": False,
"iam_role": False,
"region": False,
}
class Prior(Expression): class Prior(Expression):
pass pass
@ -2058,7 +2083,7 @@ class Insert(DDL, DML):
"hint": False, "hint": False,
"with": False, "with": False,
"is_function": False, "is_function": False,
"this": True, "this": False,
"expression": False, "expression": False,
"conflict": False, "conflict": False,
"returning": False, "returning": False,
@ -3909,6 +3934,7 @@ class DataType(Expression):
TIME = auto() TIME = auto()
TIMETZ = auto() TIMETZ = auto()
TIMESTAMP = auto() TIMESTAMP = auto()
TIMESTAMPNTZ = auto()
TIMESTAMPLTZ = auto() TIMESTAMPLTZ = auto()
TIMESTAMPTZ = auto() TIMESTAMPTZ = auto()
TIMESTAMP_S = auto() TIMESTAMP_S = auto()
@ -3936,6 +3962,7 @@ class DataType(Expression):
VARIANT = auto() VARIANT = auto()
XML = auto() XML = auto()
YEAR = auto() YEAR = auto()
TDIGEST = auto()
STRUCT_TYPES = { STRUCT_TYPES = {
Type.NESTED, Type.NESTED,
@ -4010,6 +4037,7 @@ class DataType(Expression):
Type.DATETIME64, Type.DATETIME64,
Type.TIME, Type.TIME,
Type.TIMESTAMP, Type.TIMESTAMP,
Type.TIMESTAMPNTZ,
Type.TIMESTAMPLTZ, Type.TIMESTAMPLTZ,
Type.TIMESTAMPTZ, Type.TIMESTAMPTZ,
Type.TIMESTAMP_MS, Type.TIMESTAMP_MS,
@ -4847,6 +4875,10 @@ class TryCast(Cast):
pass pass
class Try(Func):
pass
class CastToStrType(Func): class CastToStrType(Func):
arg_types = {"this": True, "to": True} arg_types = {"this": True, "to": True}
@ -5538,7 +5570,6 @@ class RegexpReplace(Func):
"replacement": False, "replacement": False,
"position": False, "position": False,
"occurrence": False, "occurrence": False,
"parameters": False,
"modifiers": False, "modifiers": False,
} }
@ -6506,7 +6537,7 @@ def and_(
**opts: other options to use to parse the input expressions. **opts: other options to use to parse the input expressions.
Returns: Returns:
And: the new condition The new condition
""" """
return t.cast(Condition, _combine(expressions, And, dialect, copy=copy, **opts)) return t.cast(Condition, _combine(expressions, And, dialect, copy=copy, **opts))
@ -6529,11 +6560,34 @@ def or_(
**opts: other options to use to parse the input expressions. **opts: other options to use to parse the input expressions.
Returns: Returns:
Or: the new condition The new condition
""" """
return t.cast(Condition, _combine(expressions, Or, dialect, copy=copy, **opts)) return t.cast(Condition, _combine(expressions, Or, dialect, copy=copy, **opts))
def xor(
*expressions: t.Optional[ExpOrStr], dialect: DialectType = None, copy: bool = True, **opts
) -> Condition:
"""
Combine multiple conditions with an XOR logical operator.
Example:
>>> xor("x=1", xor("y=1", "z=1")).sql()
'x = 1 XOR (y = 1 XOR z = 1)'
Args:
*expressions: the SQL code strings to parse.
If an Expression instance is passed, this is used as-is.
dialect: the dialect used to parse the input expression.
copy: whether to copy `expressions` (only applies to Expressions).
**opts: other options to use to parse the input expressions.
Returns:
The new condition
"""
return t.cast(Condition, _combine(expressions, Xor, dialect, copy=copy, **opts))
def not_(expression: ExpOrStr, dialect: DialectType = None, copy: bool = True, **opts) -> Not: def not_(expression: ExpOrStr, dialect: DialectType = None, copy: bool = True, **opts) -> Not:
""" """
Wrap a condition with a NOT operator. Wrap a condition with a NOT operator.

83
sqlglot/generator.py
View file

@ -339,6 +339,18 @@ class Generator(metaclass=_Generator):
# True means limit 1 happens after the union, False means it it happens on y. # True means limit 1 happens after the union, False means it it happens on y.
OUTER_UNION_MODIFIERS = True OUTER_UNION_MODIFIERS = True
# Whether parameters from COPY statement are wrapped in parentheses
COPY_PARAMS_ARE_WRAPPED = True
# Whether values of params are set with "=" token or empty space
COPY_PARAMS_EQ_REQUIRED = False
# Whether COPY statement has INTO keyword
COPY_HAS_INTO_KEYWORD = True
# Whether the conditional TRY(expression) function is supported
TRY_SUPPORTED = True
TYPE_MAPPING = { TYPE_MAPPING = {
exp.DataType.Type.NCHAR: "CHAR", exp.DataType.Type.NCHAR: "CHAR",
exp.DataType.Type.NVARCHAR: "VARCHAR", exp.DataType.Type.NVARCHAR: "VARCHAR",
@ -3158,6 +3170,13 @@ class Generator(metaclass=_Generator):
def trycast_sql(self, expression: exp.TryCast) -> str: def trycast_sql(self, expression: exp.TryCast) -> str:
return self.cast_sql(expression, safe_prefix="TRY_") return self.cast_sql(expression, safe_prefix="TRY_")
def try_sql(self, expression: exp.Try) -> str:
if not self.TRY_SUPPORTED:
self.unsupported("Unsupported TRY function")
return self.sql(expression, "this")
return self.func("TRY", expression.this)
def log_sql(self, expression: exp.Log) -> str: def log_sql(self, expression: exp.Log) -> str:
this = expression.this this = expression.this
expr = expression.expression expr = expression.expression
@ -3334,9 +3353,10 @@ class Generator(metaclass=_Generator):
then_expression = expression.args.get("then") then_expression = expression.args.get("then")
if isinstance(then_expression, exp.Insert): if isinstance(then_expression, exp.Insert):
then = f"INSERT {self.sql(then_expression, 'this')}" this = self.sql(then_expression, "this")
if "expression" in then_expression.args: this = f"INSERT {this}" if this else "INSERT"
then += f" VALUES {self.sql(then_expression, 'expression')}" then = self.sql(then_expression, "expression")
then = f"{this} VALUES {then}" if then else this
elif isinstance(then_expression, exp.Update): elif isinstance(then_expression, exp.Update):
if isinstance(then_expression.args.get("expressions"), exp.Star): if isinstance(then_expression.args.get("expressions"), exp.Star):
then = f"UPDATE {self.sql(then_expression, 'expressions')}" then = f"UPDATE {self.sql(then_expression, 'expressions')}"
@ -3358,10 +3378,11 @@ class Generator(metaclass=_Generator):
this = self.sql(table) this = self.sql(table)
using = f"USING {self.sql(expression, 'using')}" using = f"USING {self.sql(expression, 'using')}"
on = f"ON {self.sql(expression, 'on')}" on = f"ON {self.sql(expression, 'on')}"
expressions = self.expressions(expression, sep=" ") expressions = self.expressions(expression, sep=" ", indent=False)
sep = self.sep()
return self.prepend_ctes( return self.prepend_ctes(
expression, f"MERGE INTO {this}{table_alias} {using} {on} {expressions}" expression, f"MERGE INTO {this}{table_alias}{sep}{using}{sep}{on}{sep}{expressions}"
) )
def tochar_sql(self, expression: exp.ToChar) -> str: def tochar_sql(self, expression: exp.ToChar) -> str:
@ -3757,3 +3778,55 @@ class Generator(metaclass=_Generator):
if self.pretty: if self.pretty:
return string.replace("\n", self.SENTINEL_LINE_BREAK) return string.replace("\n", self.SENTINEL_LINE_BREAK)
return string return string
def copyparameter_sql(self, expression: exp.CopyParameter) -> str:
option = self.sql(expression, "this")
value = self.sql(expression, "expression")
if not value:
return option
op = " = " if self.COPY_PARAMS_EQ_REQUIRED else " "
return f"{option}{op}{value}"
def credentials_sql(self, expression: exp.Credentials) -> str:
cred_expr = expression.args.get("credentials")
if isinstance(cred_expr, exp.Literal):
# Redshift case: CREDENTIALS <string>
credentials = self.sql(expression, "credentials")
credentials = f"CREDENTIALS {credentials}" if credentials else ""
else:
# Snowflake case: CREDENTIALS = (...)
credentials = self.expressions(expression, key="credentials", flat=True, sep=" ")
credentials = f"CREDENTIALS = ({credentials})" if credentials else ""
storage = self.sql(expression, "storage")
storage = f" {storage}" if storage else ""
encryption = self.expressions(expression, key="encryption", flat=True, sep=" ")
encryption = f" ENCRYPTION = ({encryption})" if encryption else ""
iam_role = self.sql(expression, "iam_role")
iam_role = f"IAM_ROLE {iam_role}" if iam_role else ""
region = self.sql(expression, "region")
region = f" REGION {region}" if region else ""
return f"{credentials}{storage}{encryption}{iam_role}{region}"
def copy_sql(self, expression: exp.Copy) -> str:
this = self.sql(expression, "this")
this = f" INTO {this}" if self.COPY_HAS_INTO_KEYWORD else f" {this}"
credentials = self.sql(expression, "credentials")
credentials = f" {credentials}" if credentials else ""
kind = " FROM " if expression.args.get("kind") else " TO "
files = self.expressions(expression, key="files", flat=True)
sep = ", " if self.dialect.COPY_PARAMS_ARE_CSV else " "
params = self.expressions(expression, key="params", flat=True, sep=sep)
if params:
params = f" WITH ({params})" if self.COPY_PARAMS_ARE_WRAPPED else f" {params}"
return f"COPY{this}{kind}{files}{credentials}{params}"

27
sqlglot/lineage.py
View file

@ -8,6 +8,7 @@ from dataclasses import dataclass, field
from sqlglot import Schema, exp, maybe_parse from sqlglot import Schema, exp, maybe_parse
from sqlglot.errors import SqlglotError from sqlglot.errors import SqlglotError
from sqlglot.optimizer import Scope, build_scope, find_all_in_scope, normalize_identifiers, qualify from sqlglot.optimizer import Scope, build_scope, find_all_in_scope, normalize_identifiers, qualify
from sqlglot.optimizer.scope import ScopeType
if t.TYPE_CHECKING: if t.TYPE_CHECKING:
from sqlglot.dialects.dialect import DialectType from sqlglot.dialects.dialect import DialectType
@ -129,12 +130,6 @@ def to_node(
reference_node_name: t.Optional[str] = None, reference_node_name: t.Optional[str] = None,
trim_selects: bool = True, trim_selects: bool = True,
) -> Node: ) -> Node:
source_names = {
dt.alias: dt.comments[0].split()[1]
for dt in scope.derived_tables
if dt.comments and dt.comments[0].startswith("source: ")
}
# Find the specific select clause that is the source of the column we want. # Find the specific select clause that is the source of the column we want.
# This can either be a specific, named select or a generic `*` clause. # This can either be a specific, named select or a generic `*` clause.
select = ( select = (
@ -242,13 +237,31 @@ def to_node(
# If the source is a UDTF find columns used in the UTDF to generate the table # If the source is a UDTF find columns used in the UTDF to generate the table
if isinstance(source, exp.UDTF): if isinstance(source, exp.UDTF):
source_columns |= set(source.find_all(exp.Column)) source_columns |= set(source.find_all(exp.Column))
derived_tables = [
source.expression.parent
for source in scope.sources.values()
if isinstance(source, Scope) and source.is_derived_table
]
else:
derived_tables = scope.derived_tables
source_names = {
dt.alias: dt.comments[0].split()[1]
for dt in derived_tables
if dt.comments and dt.comments[0].startswith("source: ")
}
for c in source_columns: for c in source_columns:
table = c.table table = c.table
source = scope.sources.get(table) source = scope.sources.get(table)
if isinstance(source, Scope): if isinstance(source, Scope):
reference_node_name = None
if source.scope_type == ScopeType.DERIVED_TABLE and table not in source_names:
reference_node_name = table
elif source.scope_type == ScopeType.CTE:
selected_node, _ = scope.selected_sources.get(table, (None, None)) selected_node, _ = scope.selected_sources.get(table, (None, None))
reference_node_name = selected_node.name if selected_node else None
# The table itself came from a more specific scope. Recurse into that one using the unaliased column name. # The table itself came from a more specific scope. Recurse into that one using the unaliased column name.
to_node( to_node(
c.name, c.name,
@ -257,7 +270,7 @@ def to_node(
scope_name=table, scope_name=table,
upstream=node, upstream=node,
source_name=source_names.get(table) or source_name, source_name=source_names.get(table) or source_name,
reference_node_name=selected_node.name if selected_node else None, reference_node_name=reference_node_name,
trim_selects=trim_selects, trim_selects=trim_selects,
) )
else: else:

14
sqlglot/optimizer/__init__.py
View file

@ -1,11 +1,11 @@
# ruff: noqa: F401 # ruff: noqa: F401
from sqlglot.optimizer.optimizer import RULES, optimize from sqlglot.optimizer.optimizer import RULES as RULES, optimize as optimize
from sqlglot.optimizer.scope import ( from sqlglot.optimizer.scope import (
Scope, Scope as Scope,
build_scope, build_scope as build_scope,
find_all_in_scope, find_all_in_scope as find_all_in_scope,
find_in_scope, find_in_scope as find_in_scope,
traverse_scope, traverse_scope as traverse_scope,
walk_in_scope, walk_in_scope as walk_in_scope,
) )

3
sqlglot/optimizer/eliminate_subqueries.py
View file

@ -89,7 +89,8 @@ def eliminate_subqueries(expression):
new_ctes.append(new_cte) new_ctes.append(new_cte)
if new_ctes: if new_ctes:
expression.set("with", exp.With(expressions=new_ctes, recursive=recursive)) query = expression.expression if isinstance(expression, exp.DDL) else expression
query.set("with", exp.With(expressions=new_ctes, recursive=recursive))
return expression return expression

54
sqlglot/optimizer/scope.py
View file

@ -12,6 +12,8 @@ from sqlglot.helper import ensure_collection, find_new_name, seq_get
logger = logging.getLogger("sqlglot") logger = logging.getLogger("sqlglot")
TRAVERSABLES = (exp.Query, exp.DDL, exp.DML)
class ScopeType(Enum): class ScopeType(Enum):
ROOT = auto() ROOT = auto()
@ -495,25 +497,8 @@ def traverse_scope(expression: exp.Expression) -> t.List[Scope]:
Returns: Returns:
A list of the created scope instances A list of the created scope instances
""" """
if isinstance(expression, exp.DDL) and isinstance(expression.expression, exp.Query): if isinstance(expression, TRAVERSABLES):
# We ignore the DDL expression and build a scope for its query instead
ddl_with = expression.args.get("with")
expression = expression.expression
# If the DDL has CTEs attached, we need to add them to the query, or
# prepend them if the query itself already has CTEs attached to it
if ddl_with:
ddl_with.pop()
query_ctes = expression.ctes
if not query_ctes:
expression.set("with", ddl_with)
else:
expression.args["with"].set("recursive", ddl_with.recursive)
expression.args["with"].set("expressions", [*ddl_with.expressions, *query_ctes])
if isinstance(expression, exp.Query):
return list(_traverse_scope(Scope(expression))) return list(_traverse_scope(Scope(expression)))
return [] return []
@ -531,25 +516,37 @@ def build_scope(expression: exp.Expression) -> t.Optional[Scope]:
def _traverse_scope(scope): def _traverse_scope(scope):
if isinstance(scope.expression, exp.Select): expression = scope.expression
if isinstance(expression, exp.Select):
yield from _traverse_select(scope) yield from _traverse_select(scope)
elif isinstance(scope.expression, exp.Union): elif isinstance(expression, exp.Union):
yield from _traverse_ctes(scope) yield from _traverse_ctes(scope)
yield from _traverse_union(scope) yield from _traverse_union(scope)
return return
elif isinstance(scope.expression, exp.Subquery): elif isinstance(expression, exp.Subquery):
if scope.is_root: if scope.is_root:
yield from _traverse_select(scope) yield from _traverse_select(scope)
else: else:
yield from _traverse_subqueries(scope) yield from _traverse_subqueries(scope)
elif isinstance(scope.expression, exp.Table): elif isinstance(expression, exp.Table):
yield from _traverse_tables(scope) yield from _traverse_tables(scope)
elif isinstance(scope.expression, exp.UDTF): elif isinstance(expression, exp.UDTF):
yield from _traverse_udtfs(scope) yield from _traverse_udtfs(scope)
elif isinstance(expression, exp.DDL):
if isinstance(expression.expression, exp.Query):
yield from _traverse_ctes(scope)
yield from _traverse_scope(Scope(expression.expression, cte_sources=scope.cte_sources))
return
elif isinstance(expression, exp.DML):
yield from _traverse_ctes(scope)
for query in find_all_in_scope(expression, exp.Query):
# This check ensures we don't yield the CTE queries twice
if not isinstance(query.parent, exp.CTE):
yield from _traverse_scope(Scope(query, cte_sources=scope.cte_sources))
return
else: else:
logger.warning( logger.warning("Cannot traverse scope %s with type '%s'", expression, type(expression))
"Cannot traverse scope %s with type '%s'", scope.expression, type(scope.expression)
)
return return
yield scope yield scope
@ -749,7 +746,7 @@ def _traverse_udtfs(scope):
for child_scope in _traverse_scope( for child_scope in _traverse_scope(
scope.branch( scope.branch(
expression, expression,
scope_type=ScopeType.DERIVED_TABLE, scope_type=ScopeType.SUBQUERY,
outer_columns=expression.alias_column_names, outer_columns=expression.alias_column_names,
) )
): ):
@ -757,8 +754,7 @@ def _traverse_udtfs(scope):
top = child_scope top = child_scope
sources[expression.alias] = child_scope sources[expression.alias] = child_scope
scope.derived_table_scopes.append(top) scope.subquery_scopes.append(top)
scope.table_scopes.append(top)
scope.sources.update(sources) scope.sources.update(sources)

13
sqlglot/optimizer/simplify.py
View file

@ -224,6 +224,8 @@ def flatten(expression):
def simplify_connectors(expression, root=True): def simplify_connectors(expression, root=True):
def _simplify_connectors(expression, left, right): def _simplify_connectors(expression, left, right):
if left == right: if left == right:
if isinstance(expression, exp.Xor):
return exp.false()
return left return left
if isinstance(expression, exp.And): if isinstance(expression, exp.And):
if is_false(left) or is_false(right): if is_false(left) or is_false(right):
@ -365,8 +367,15 @@ def uniq_sort(expression, root=True):
C AND A AND B AND B -> A AND B AND C C AND A AND B AND B -> A AND B AND C
""" """
if isinstance(expression, exp.Connector) and (root or not expression.same_parent): if isinstance(expression, exp.Connector) and (root or not expression.same_parent):
result_func = exp.and_ if isinstance(expression, exp.And) else exp.or_
flattened = tuple(expression.flatten()) flattened = tuple(expression.flatten())
if isinstance(expression, exp.Xor):
result_func = exp.xor
# Do not deduplicate XOR as A XOR A != A if A == True
deduped = None
arr = tuple((gen(e), e) for e in flattened)
else:
result_func = exp.and_ if isinstance(expression, exp.And) else exp.or_
deduped = {gen(e): e for e in flattened} deduped = {gen(e): e for e in flattened}
arr = tuple(deduped.items()) arr = tuple(deduped.items())
@ -378,7 +387,7 @@ def uniq_sort(expression, root=True):
break break
else: else:
# we didn't have to sort but maybe we need to dedup # we didn't have to sort but maybe we need to dedup
if len(deduped) < len(flattened): if deduped and len(deduped) < len(flattened):
expression = result_func(*deduped.values(), copy=False) expression = result_func(*deduped.values(), copy=False)
return expression return expression

146
sqlglot/parser.py
View file

@ -217,6 +217,7 @@ class Parser(metaclass=_Parser):
TokenType.TIMESTAMP_NS, TokenType.TIMESTAMP_NS,
TokenType.TIMESTAMPTZ, TokenType.TIMESTAMPTZ,
TokenType.TIMESTAMPLTZ, TokenType.TIMESTAMPLTZ,
TokenType.TIMESTAMPNTZ,
TokenType.DATETIME, TokenType.DATETIME,
TokenType.DATETIME64, TokenType.DATETIME64,
TokenType.DATE, TokenType.DATE,
@ -265,6 +266,7 @@ class Parser(metaclass=_Parser):
TokenType.UNKNOWN, TokenType.UNKNOWN,
TokenType.NULL, TokenType.NULL,
TokenType.NAME, TokenType.NAME,
TokenType.TDIGEST,
*ENUM_TYPE_TOKENS, *ENUM_TYPE_TOKENS,
*NESTED_TYPE_TOKENS, *NESTED_TYPE_TOKENS,
*AGGREGATE_TYPE_TOKENS, *AGGREGATE_TYPE_TOKENS,
@ -329,6 +331,7 @@ class Parser(metaclass=_Parser):
TokenType.COMMENT, TokenType.COMMENT,
TokenType.COMMIT, TokenType.COMMIT,
TokenType.CONSTRAINT, TokenType.CONSTRAINT,
TokenType.COPY,
TokenType.DEFAULT, TokenType.DEFAULT,
TokenType.DELETE, TokenType.DELETE,
TokenType.DESC, TokenType.DESC,
@ -597,7 +600,7 @@ class Parser(metaclass=_Parser):
exp.Condition: lambda self: self._parse_conjunction(), exp.Condition: lambda self: self._parse_conjunction(),
exp.DataType: lambda self: self._parse_types(allow_identifiers=False), exp.DataType: lambda self: self._parse_types(allow_identifiers=False),
exp.Expression: lambda self: self._parse_expression(), exp.Expression: lambda self: self._parse_expression(),
exp.From: lambda self: self._parse_from(), exp.From: lambda self: self._parse_from(joins=True),
exp.Group: lambda self: self._parse_group(), exp.Group: lambda self: self._parse_group(),
exp.Having: lambda self: self._parse_having(), exp.Having: lambda self: self._parse_having(),
exp.Identifier: lambda self: self._parse_id_var(), exp.Identifier: lambda self: self._parse_id_var(),
@ -627,6 +630,7 @@ class Parser(metaclass=_Parser):
TokenType.CACHE: lambda self: self._parse_cache(), TokenType.CACHE: lambda self: self._parse_cache(),
TokenType.COMMENT: lambda self: self._parse_comment(), TokenType.COMMENT: lambda self: self._parse_comment(),
TokenType.COMMIT: lambda self: self._parse_commit_or_rollback(), TokenType.COMMIT: lambda self: self._parse_commit_or_rollback(),
TokenType.COPY: lambda self: self._parse_copy(),
TokenType.CREATE: lambda self: self._parse_create(), TokenType.CREATE: lambda self: self._parse_create(),
TokenType.DELETE: lambda self: self._parse_delete(), TokenType.DELETE: lambda self: self._parse_delete(),
TokenType.DESC: lambda self: self._parse_describe(), TokenType.DESC: lambda self: self._parse_describe(),
@ -1585,7 +1589,15 @@ class Parser(metaclass=_Parser):
if return_: if return_:
expression = self.expression(exp.Return, this=expression) expression = self.expression(exp.Return, this=expression)
elif create_token.token_type == TokenType.INDEX: elif create_token.token_type == TokenType.INDEX:
this = self._parse_index(index=self._parse_id_var()) # Postgres allows anonymous indexes, eg. CREATE INDEX IF NOT EXISTS ON t(c)
if not self._match(TokenType.ON):
index = self._parse_id_var()
anonymous = False
else:
index = None
anonymous = True
this = self._parse_index(index=index, anonymous=anonymous)
elif create_token.token_type in self.DB_CREATABLES: elif create_token.token_type in self.DB_CREATABLES:
table_parts = self._parse_table_parts( table_parts = self._parse_table_parts(
schema=True, is_db_reference=create_token.token_type == TokenType.SCHEMA schema=True, is_db_reference=create_token.token_type == TokenType.SCHEMA
@ -1764,14 +1776,18 @@ class Parser(metaclass=_Parser):
), ),
) )
def _parse_property_assignment(self, exp_class: t.Type[E], **kwargs: t.Any) -> E: def _parse_unquoted_field(self):
self._match(TokenType.EQ)
self._match(TokenType.ALIAS)
field = self._parse_field() field = self._parse_field()
if isinstance(field, exp.Identifier) and not field.quoted: if isinstance(field, exp.Identifier) and not field.quoted:
field = exp.var(field) field = exp.var(field)
return self.expression(exp_class, this=field, **kwargs) return field
def _parse_property_assignment(self, exp_class: t.Type[E], **kwargs: t.Any) -> E:
self._match(TokenType.EQ)
self._match(TokenType.ALIAS)
return self.expression(exp_class, this=self._parse_unquoted_field(), **kwargs)
def _parse_properties(self, before: t.Optional[bool] = None) -> t.Optional[exp.Properties]: def _parse_properties(self, before: t.Optional[bool] = None) -> t.Optional[exp.Properties]:
properties = [] properties = []
@ -2206,9 +2222,9 @@ class Parser(metaclass=_Parser):
def _parse_describe(self) -> exp.Describe: def _parse_describe(self) -> exp.Describe:
kind = self._match_set(self.CREATABLES) and self._prev.text kind = self._match_set(self.CREATABLES) and self._prev.text
style = self._match_texts(("EXTENDED", "FORMATTED", "HISTORY")) and self._prev.text.upper() style = self._match_texts(("EXTENDED", "FORMATTED", "HISTORY")) and self._prev.text.upper()
if not self._match_set(self.ID_VAR_TOKENS, advance=False): if self._match(TokenType.DOT):
style = None style = None
self._retreat(self._index - 1) self._retreat(self._index - 2)
this = self._parse_table(schema=True) this = self._parse_table(schema=True)
properties = self._parse_properties() properties = self._parse_properties()
expressions = properties.expressions if properties else None expressions = properties.expressions if properties else None
@ -2461,14 +2477,17 @@ class Parser(metaclass=_Parser):
exp.Partition, expressions=self._parse_wrapped_csv(self._parse_conjunction) exp.Partition, expressions=self._parse_wrapped_csv(self._parse_conjunction)
) )
def _parse_value(self) -> exp.Tuple: def _parse_value(self) -> t.Optional[exp.Tuple]:
if self._match(TokenType.L_PAREN): if self._match(TokenType.L_PAREN):
expressions = self._parse_csv(self._parse_expression) expressions = self._parse_csv(self._parse_expression)
self._match_r_paren() self._match_r_paren()
return self.expression(exp.Tuple, expressions=expressions) return self.expression(exp.Tuple, expressions=expressions)
# In some dialects we can have VALUES 1, 2 which results in 1 column & 2 rows. # In some dialects we can have VALUES 1, 2 which results in 1 column & 2 rows.
return self.expression(exp.Tuple, expressions=[self._parse_expression()]) expression = self._parse_expression()
if expression:
return self.expression(exp.Tuple, expressions=[expression])
return None
def _parse_projections(self) -> t.List[exp.Expression]: def _parse_projections(self) -> t.List[exp.Expression]:
return self._parse_expressions() return self._parse_expressions()
@ -3010,10 +3029,9 @@ class Parser(metaclass=_Parser):
) )
def _parse_index( def _parse_index(
self, self, index: t.Optional[exp.Expression] = None, anonymous: bool = False
index: t.Optional[exp.Expression] = None,
) -> t.Optional[exp.Index]: ) -> t.Optional[exp.Index]:
if index: if index or anonymous:
unique = None unique = None
primary = None primary = None
amp = None amp = None
@ -4305,7 +4323,9 @@ class Parser(metaclass=_Parser):
this = self._parse_query_modifiers(seq_get(expressions, 0)) this = self._parse_query_modifiers(seq_get(expressions, 0))
if isinstance(this, exp.UNWRAPPED_QUERIES): if not this and self._match(TokenType.R_PAREN, advance=False):
this = self.expression(exp.Tuple)
elif isinstance(this, exp.UNWRAPPED_QUERIES):
this = self._parse_set_operations( this = self._parse_set_operations(
self._parse_subquery(this=this, parse_alias=False) self._parse_subquery(this=this, parse_alias=False)
) )
@ -4675,7 +4695,7 @@ class Parser(metaclass=_Parser):
this.set("cycle", False) this.set("cycle", False)
if not identity: if not identity:
this.set("expression", self._parse_bitwise()) this.set("expression", self._parse_range())
elif not this.args.get("start") and self._match(TokenType.NUMBER, advance=False): elif not this.args.get("start") and self._match(TokenType.NUMBER, advance=False):
args = self._parse_csv(self._parse_bitwise) args = self._parse_csv(self._parse_bitwise)
this.set("start", seq_get(args, 0)) this.set("start", seq_get(args, 0))
@ -5310,6 +5330,8 @@ class Parser(metaclass=_Parser):
if self._match(TokenType.FROM): if self._match(TokenType.FROM):
args.append(self._parse_bitwise()) args.append(self._parse_bitwise())
if self._match(TokenType.FOR): if self._match(TokenType.FOR):
if len(args) == 1:
args.append(exp.Literal.number(1))
args.append(self._parse_bitwise()) args.append(self._parse_bitwise())
return self.validate_expression(exp.Substring.from_arg_list(args), args) return self.validate_expression(exp.Substring.from_arg_list(args), args)
@ -6292,3 +6314,97 @@ class Parser(metaclass=_Parser):
op = self._parse_var(any_token=True) op = self._parse_var(any_token=True)
return self.expression(exp.WithOperator, this=this, op=op) return self.expression(exp.WithOperator, this=this, op=op)
def _parse_wrapped_options(self) -> t.List[t.Optional[exp.Expression]]:
opts = []
self._match(TokenType.EQ)
self._match(TokenType.L_PAREN)
while self._curr and not self._match(TokenType.R_PAREN):
opts.append(self._parse_conjunction())
self._match(TokenType.COMMA)
return opts
def _parse_copy_parameters(self) -> t.List[exp.CopyParameter]:
sep = TokenType.COMMA if self.dialect.COPY_PARAMS_ARE_CSV else None
options = []
while self._curr and not self._match(TokenType.R_PAREN, advance=False):
option = self._parse_unquoted_field()
value = None
# Some options are defined as functions with the values as params
if not isinstance(option, exp.Func):
prev = self._prev.text.upper()
# Different dialects might separate options and values by white space, "=" and "AS"
self._match(TokenType.EQ)
self._match(TokenType.ALIAS)
if prev == "FILE_FORMAT" and self._match(TokenType.L_PAREN):
# Snowflake FILE_FORMAT case
value = self._parse_wrapped_options()
else:
value = self._parse_unquoted_field()
param = self.expression(exp.CopyParameter, this=option, expression=value)
options.append(param)
if sep:
self._match(sep)
return options
def _parse_credentials(self) -> t.Optional[exp.Credentials]:
expr = self.expression(exp.Credentials)
if self._match_text_seq("STORAGE_INTEGRATION", advance=False):
expr.set("storage", self._parse_conjunction())
if self._match_text_seq("CREDENTIALS"):
# Snowflake supports CREDENTIALS = (...), while Redshift CREDENTIALS <string>
creds = (
self._parse_wrapped_options() if self._match(TokenType.EQ) else self._parse_field()
)
expr.set("credentials", creds)
if self._match_text_seq("ENCRYPTION"):
expr.set("encryption", self._parse_wrapped_options())
if self._match_text_seq("IAM_ROLE"):
expr.set("iam_role", self._parse_field())
if self._match_text_seq("REGION"):
expr.set("region", self._parse_field())
return expr
def _parse_file_location(self) -> t.Optional[exp.Expression]:
return self._parse_field()
def _parse_copy(self) -> exp.Copy | exp.Command:
start = self._prev
self._match(TokenType.INTO)
this = (
self._parse_conjunction()
if self._match(TokenType.L_PAREN, advance=False)
else self._parse_table(schema=True)
)
kind = self._match(TokenType.FROM) or not self._match_text_seq("TO")
files = self._parse_csv(self._parse_file_location)
credentials = self._parse_credentials()
self._match_text_seq("WITH")
params = self._parse_wrapped(self._parse_copy_parameters, optional=True)
# Fallback case
if self._curr:
return self._parse_as_command(start)
return self.expression(
exp.Copy,
this=this,
kind=kind,
credentials=credentials,
files=files,
params=params,
)

8
sqlglot/tokens.py
View file

@ -145,6 +145,7 @@ class TokenType(AutoName):
TIMESTAMP = auto() TIMESTAMP = auto()
TIMESTAMPTZ = auto() TIMESTAMPTZ = auto()
TIMESTAMPLTZ = auto() TIMESTAMPLTZ = auto()
TIMESTAMPNTZ = auto()
TIMESTAMP_S = auto() TIMESTAMP_S = auto()
TIMESTAMP_MS = auto() TIMESTAMP_MS = auto()
TIMESTAMP_NS = auto() TIMESTAMP_NS = auto()
@ -197,6 +198,7 @@ class TokenType(AutoName):
NESTED = auto() NESTED = auto()
AGGREGATEFUNCTION = auto() AGGREGATEFUNCTION = auto()
SIMPLEAGGREGATEFUNCTION = auto() SIMPLEAGGREGATEFUNCTION = auto()
TDIGEST = auto()
UNKNOWN = auto() UNKNOWN = auto()
# keywords # keywords
@ -223,6 +225,7 @@ class TokenType(AutoName):
COMMIT = auto() COMMIT = auto()
CONNECT_BY = auto() CONNECT_BY = auto()
CONSTRAINT = auto() CONSTRAINT = auto()
COPY = auto()
CREATE = auto() CREATE = auto()
CROSS = auto() CROSS = auto()
CUBE = auto() CUBE = auto()
@ -647,6 +650,7 @@ class Tokenizer(metaclass=_Tokenizer):
"COMMIT": TokenType.COMMIT, "COMMIT": TokenType.COMMIT,
"CONNECT BY": TokenType.CONNECT_BY, "CONNECT BY": TokenType.CONNECT_BY,
"CONSTRAINT": TokenType.CONSTRAINT, "CONSTRAINT": TokenType.CONSTRAINT,
"COPY": TokenType.COPY,
"CREATE": TokenType.CREATE, "CREATE": TokenType.CREATE,
"CROSS": TokenType.CROSS, "CROSS": TokenType.CROSS,
"CUBE": TokenType.CUBE, "CUBE": TokenType.CUBE,
@ -845,6 +849,9 @@ class Tokenizer(metaclass=_Tokenizer):
"TIMESTAMP": TokenType.TIMESTAMP, "TIMESTAMP": TokenType.TIMESTAMP,
"TIMESTAMPTZ": TokenType.TIMESTAMPTZ, "TIMESTAMPTZ": TokenType.TIMESTAMPTZ,
"TIMESTAMPLTZ": TokenType.TIMESTAMPLTZ, "TIMESTAMPLTZ": TokenType.TIMESTAMPLTZ,
"TIMESTAMP_LTZ": TokenType.TIMESTAMPLTZ,
"TIMESTAMPNTZ": TokenType.TIMESTAMPNTZ,
"TIMESTAMP_NTZ": TokenType.TIMESTAMPNTZ,
"DATE": TokenType.DATE, "DATE": TokenType.DATE,
"DATETIME": TokenType.DATETIME, "DATETIME": TokenType.DATETIME,
"INT4RANGE": TokenType.INT4RANGE, "INT4RANGE": TokenType.INT4RANGE,
@ -867,7 +874,6 @@ class Tokenizer(metaclass=_Tokenizer):
"ANALYZE": TokenType.COMMAND, "ANALYZE": TokenType.COMMAND,
"CALL": TokenType.COMMAND, "CALL": TokenType.COMMAND,
"COMMENT": TokenType.COMMENT, "COMMENT": TokenType.COMMENT,
"COPY": TokenType.COMMAND,
"EXPLAIN": TokenType.COMMAND, "EXPLAIN": TokenType.COMMAND,
"GRANT": TokenType.COMMAND, "GRANT": TokenType.COMMAND,
"OPTIMIZE": TokenType.COMMAND, "OPTIMIZE": TokenType.COMMAND,

6
tests/dialects/test_databricks.py
View file

@ -25,6 +25,9 @@ class TestDatabricks(Validator):
self.validate_identity("CREATE FUNCTION a AS b") self.validate_identity("CREATE FUNCTION a AS b")
self.validate_identity("SELECT ${x} FROM ${y} WHERE ${z} > 1") self.validate_identity("SELECT ${x} FROM ${y} WHERE ${z} > 1")
self.validate_identity("CREATE TABLE foo (x DATE GENERATED ALWAYS AS (CAST(y AS DATE)))") self.validate_identity("CREATE TABLE foo (x DATE GENERATED ALWAYS AS (CAST(y AS DATE)))")
self.validate_identity(
"CREATE TABLE IF NOT EXISTS db.table (a TIMESTAMP, b BOOLEAN GENERATED ALWAYS AS (NOT a IS NULL)) USING DELTA"
)
self.validate_identity( self.validate_identity(
"SELECT DATE_FORMAT(CAST(FROM_UTC_TIMESTAMP(CAST(foo AS TIMESTAMP), 'America/Los_Angeles') AS TIMESTAMP), 'yyyy-MM-dd HH:mm:ss') AS foo FROM t" "SELECT DATE_FORMAT(CAST(FROM_UTC_TIMESTAMP(CAST(foo AS TIMESTAMP), 'America/Los_Angeles') AS TIMESTAMP), 'yyyy-MM-dd HH:mm:ss') AS foo FROM t"
) )
@ -47,6 +50,9 @@ class TestDatabricks(Validator):
self.validate_identity( self.validate_identity(
"TRUNCATE TABLE t1 PARTITION(age = 10, name = 'test', city LIKE 'LA')" "TRUNCATE TABLE t1 PARTITION(age = 10, name = 'test', city LIKE 'LA')"
) )
self.validate_identity(
"COPY INTO target FROM `s3://link` FILEFORMAT = AVRO VALIDATE = ALL FILES = ('file1', 'file2') FORMAT_OPTIONS(opt1 = TRUE, opt2 = 'test') COPY_OPTIONS(opt3 = 5)"
)
self.validate_all( self.validate_all(
"CREATE TABLE foo (x INT GENERATED ALWAYS AS (YEAR(y)))", "CREATE TABLE foo (x INT GENERATED ALWAYS AS (YEAR(y)))",

2
tests/dialects/test_dialect.py
View file

@ -293,7 +293,7 @@ class TestDialect(Validator):
"bigquery": "CAST(a AS INT64)", "bigquery": "CAST(a AS INT64)",
"drill": "CAST(a AS INTEGER)", "drill": "CAST(a AS INTEGER)",
"duckdb": "CAST(a AS SMALLINT)", "duckdb": "CAST(a AS SMALLINT)",
"mysql": "CAST(a AS SMALLINT)", "mysql": "CAST(a AS SIGNED)",
"hive": "CAST(a AS SMALLINT)", "hive": "CAST(a AS SMALLINT)",
"oracle": "CAST(a AS NUMBER)", "oracle": "CAST(a AS NUMBER)",
"postgres": "CAST(a AS SMALLINT)", "postgres": "CAST(a AS SMALLINT)",

10
tests/dialects/test_duckdb.py
View file

@ -324,6 +324,8 @@ class TestDuckDB(Validator):
self.validate_identity( self.validate_identity(
"SELECT * FROM (PIVOT Cities ON Year USING SUM(Population) GROUP BY Country) AS pivot_alias" "SELECT * FROM (PIVOT Cities ON Year USING SUM(Population) GROUP BY Country) AS pivot_alias"
) )
self.validate_identity("DATE_SUB('YEAR', col, '2020-01-01')").assert_is(exp.Anonymous)
self.validate_identity("DATESUB('YEAR', col, '2020-01-01')").assert_is(exp.Anonymous)
self.validate_all("0b1010", write={"": "0 AS b1010"}) self.validate_all("0b1010", write={"": "0 AS b1010"})
self.validate_all("0x1010", write={"": "0 AS x1010"}) self.validate_all("0x1010", write={"": "0 AS x1010"})
@ -724,6 +726,14 @@ class TestDuckDB(Validator):
"""SELECT i FROM GENERATE_SERIES(0, 12) AS _(i) ORDER BY i ASC""", """SELECT i FROM GENERATE_SERIES(0, 12) AS _(i) ORDER BY i ASC""",
) )
self.validate_identity(
"COPY lineitem FROM 'lineitem.ndjson' WITH (FORMAT JSON, DELIMITER ',', AUTO_DETECT TRUE, COMPRESSION SNAPPY, CODEC ZSTD, FORCE_NOT_NULL(col1, col2))"
)
self.validate_identity(
"COPY (SELECT 42 AS a, 'hello' AS b) TO 'query.json' WITH (FORMAT JSON, ARRAY TRUE)"
)
self.validate_identity("COPY lineitem (l_orderkey) TO 'orderkey.tbl' WITH (DELIMITER '|')")
def test_array_index(self): def test_array_index(self):
with self.assertLogs(helper_logger) as cm: with self.assertLogs(helper_logger) as cm:
self.validate_all( self.validate_all(

4
tests/dialects/test_hive.py
View file

@ -334,7 +334,7 @@ class TestHive(Validator):
"hive": "DATE_ADD('2020-01-01', 1)", "hive": "DATE_ADD('2020-01-01', 1)",
"presto": "DATE_ADD('DAY', 1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))", "presto": "DATE_ADD('DAY', 1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))",
"redshift": "DATEADD(DAY, 1, '2020-01-01')", "redshift": "DATEADD(DAY, 1, '2020-01-01')",
"snowflake": "DATEADD(DAY, 1, CAST(CAST('2020-01-01' AS TIMESTAMPNTZ) AS DATE))", "snowflake": "DATEADD(DAY, 1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))",
"spark": "DATE_ADD('2020-01-01', 1)", "spark": "DATE_ADD('2020-01-01', 1)",
"tsql": "DATEADD(DAY, 1, CAST(CAST('2020-01-01' AS DATETIME2) AS DATE))", "tsql": "DATEADD(DAY, 1, CAST(CAST('2020-01-01' AS DATETIME2) AS DATE))",
}, },
@ -348,7 +348,7 @@ class TestHive(Validator):
"hive": "DATE_ADD('2020-01-01', 1 * -1)", "hive": "DATE_ADD('2020-01-01', 1 * -1)",
"presto": "DATE_ADD('DAY', 1 * -1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))", "presto": "DATE_ADD('DAY', 1 * -1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))",
"redshift": "DATEADD(DAY, 1 * -1, '2020-01-01')", "redshift": "DATEADD(DAY, 1 * -1, '2020-01-01')",
"snowflake": "DATEADD(DAY, 1 * -1, CAST(CAST('2020-01-01' AS TIMESTAMPNTZ) AS DATE))", "snowflake": "DATEADD(DAY, 1 * -1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))",
"spark": "DATE_ADD('2020-01-01', 1 * -1)", "spark": "DATE_ADD('2020-01-01', 1 * -1)",
"tsql": "DATEADD(DAY, 1 * -1, CAST(CAST('2020-01-01' AS DATETIME2) AS DATE))", "tsql": "DATEADD(DAY, 1 * -1, CAST(CAST('2020-01-01' AS DATETIME2) AS DATE))",
}, },

118
tests/dialects/test_mysql.py
View file

@ -1,4 +1,5 @@
from sqlglot import expressions as exp from sqlglot import expressions as exp
from sqlglot.dialects.mysql import MySQL
from tests.dialects.test_dialect import Validator from tests.dialects.test_dialect import Validator
@ -6,21 +7,11 @@ class TestMySQL(Validator):
dialect = "mysql" dialect = "mysql"
def test_ddl(self): def test_ddl(self):
int_types = {"BIGINT", "INT", "MEDIUMINT", "SMALLINT", "TINYINT"} for t in ("BIGINT", "INT", "MEDIUMINT", "SMALLINT", "TINYINT"):
for t in int_types:
self.validate_identity(f"CREATE TABLE t (id {t} UNSIGNED)") self.validate_identity(f"CREATE TABLE t (id {t} UNSIGNED)")
self.validate_identity(f"CREATE TABLE t (id {t}(10) UNSIGNED)") self.validate_identity(f"CREATE TABLE t (id {t}(10) UNSIGNED)")
self.validate_identity("CREATE TABLE t (id DECIMAL(20, 4) UNSIGNED)") self.validate_identity("CREATE TABLE t (id DECIMAL(20, 4) UNSIGNED)")
self.validate_all(
"CREATE TABLE t (id INT UNSIGNED)",
write={
"duckdb": "CREATE TABLE t (id UINTEGER)",
},
)
self.validate_identity("CREATE TABLE foo (a BIGINT, UNIQUE (b) USING BTREE)") self.validate_identity("CREATE TABLE foo (a BIGINT, UNIQUE (b) USING BTREE)")
self.validate_identity("CREATE TABLE foo (id BIGINT)") self.validate_identity("CREATE TABLE foo (id BIGINT)")
self.validate_identity("CREATE TABLE 00f (1d BIGINT)") self.validate_identity("CREATE TABLE 00f (1d BIGINT)")
@ -97,6 +88,13 @@ class TestMySQL(Validator):
"CREATE TABLE `foo` (a VARCHAR(10), INDEX idx_a (a DESC))", "CREATE TABLE `foo` (a VARCHAR(10), INDEX idx_a (a DESC))",
) )
self.validate_all(
"CREATE TABLE t (id INT UNSIGNED)",
write={
"duckdb": "CREATE TABLE t (id UINTEGER)",
"mysql": "CREATE TABLE t (id INT UNSIGNED)",
},
)
self.validate_all( self.validate_all(
"CREATE TABLE z (a INT) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARACTER SET=utf8 COLLATE=utf8_bin COMMENT='x'", "CREATE TABLE z (a INT) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARACTER SET=utf8 COLLATE=utf8_bin COMMENT='x'",
write={ write={
@ -109,15 +107,10 @@ class TestMySQL(Validator):
self.validate_all( self.validate_all(
"CREATE TABLE x (id int not null auto_increment, primary key (id))", "CREATE TABLE x (id int not null auto_increment, primary key (id))",
write={ write={
"mysql": "CREATE TABLE x (id INT NOT NULL AUTO_INCREMENT, PRIMARY KEY (id))",
"sqlite": "CREATE TABLE x (id INTEGER NOT NULL AUTOINCREMENT PRIMARY KEY)", "sqlite": "CREATE TABLE x (id INTEGER NOT NULL AUTOINCREMENT PRIMARY KEY)",
}, },
) )
self.validate_all(
"CREATE TABLE x (id int not null auto_increment)",
write={
"sqlite": "CREATE TABLE x (id INTEGER NOT NULL)",
},
)
def test_identity(self): def test_identity(self):
self.validate_identity("ALTER TABLE test_table ALTER COLUMN test_column SET DEFAULT 1") self.validate_identity("ALTER TABLE test_table ALTER COLUMN test_column SET DEFAULT 1")
@ -135,8 +128,6 @@ class TestMySQL(Validator):
self.validate_identity("SELECT CAST('[4,5]' AS JSON) MEMBER OF('[[3,4],[4,5]]')") self.validate_identity("SELECT CAST('[4,5]' AS JSON) MEMBER OF('[[3,4],[4,5]]')")
self.validate_identity("""SELECT 'ab' MEMBER OF('[23, "abc", 17, "ab", 10]')""") self.validate_identity("""SELECT 'ab' MEMBER OF('[23, "abc", 17, "ab", 10]')""")
self.validate_identity("""SELECT * FROM foo WHERE 'ab' MEMBER OF(content)""") self.validate_identity("""SELECT * FROM foo WHERE 'ab' MEMBER OF(content)""")
self.validate_identity("CAST(x AS ENUM('a', 'b'))")
self.validate_identity("CAST(x AS SET('a', 'b'))")
self.validate_identity("SELECT CURRENT_TIMESTAMP(6)") self.validate_identity("SELECT CURRENT_TIMESTAMP(6)")
self.validate_identity("x ->> '$.name'") self.validate_identity("x ->> '$.name'")
self.validate_identity("SELECT CAST(`a`.`b` AS CHAR) FROM foo") self.validate_identity("SELECT CAST(`a`.`b` AS CHAR) FROM foo")
@ -226,29 +217,47 @@ class TestMySQL(Validator):
self.validate_identity("SELECT * FROM t1 PARTITION(p0)") self.validate_identity("SELECT * FROM t1 PARTITION(p0)")
def test_types(self): def test_types(self):
self.validate_identity("CAST(x AS MEDIUMINT) + CAST(y AS YEAR(4))") for char_type in MySQL.Generator.CHAR_CAST_MAPPING:
with self.subTest(f"MySQL cast into {char_type}"):
self.validate_identity(f"CAST(x AS {char_type.value})", "CAST(x AS CHAR)")
for signed_type in MySQL.Generator.SIGNED_CAST_MAPPING:
with self.subTest(f"MySQL cast into {signed_type}"):
self.validate_identity(f"CAST(x AS {signed_type.value})", "CAST(x AS SIGNED)")
self.validate_identity("CAST(x AS ENUM('a', 'b'))")
self.validate_identity("CAST(x AS SET('a', 'b'))")
self.validate_identity(
"CAST(x AS MEDIUMINT) + CAST(y AS YEAR(4))",
"CAST(x AS SIGNED) + CAST(y AS YEAR(4))",
)
self.validate_identity(
"CAST(x AS TIMESTAMP)",
"CAST(x AS DATETIME)",
)
self.validate_identity(
"CAST(x AS TIMESTAMPTZ)",
"TIMESTAMP(x)",
)
self.validate_identity(
"CAST(x AS TIMESTAMPLTZ)",
"TIMESTAMP(x)",
)
self.validate_all( self.validate_all(
"CAST(x AS MEDIUMTEXT) + CAST(y AS LONGTEXT) + CAST(z AS TINYTEXT)", "CAST(x AS MEDIUMTEXT) + CAST(y AS LONGTEXT) + CAST(z AS TINYTEXT)",
read={
"mysql": "CAST(x AS MEDIUMTEXT) + CAST(y AS LONGTEXT) + CAST(z AS TINYTEXT)",
},
write={ write={
"mysql": "CAST(x AS CHAR) + CAST(y AS CHAR) + CAST(z AS CHAR)",
"spark": "CAST(x AS TEXT) + CAST(y AS TEXT) + CAST(z AS TEXT)", "spark": "CAST(x AS TEXT) + CAST(y AS TEXT) + CAST(z AS TEXT)",
}, },
) )
self.validate_all( self.validate_all(
"CAST(x AS MEDIUMBLOB) + CAST(y AS LONGBLOB) + CAST(z AS TINYBLOB)", "CAST(x AS MEDIUMBLOB) + CAST(y AS LONGBLOB) + CAST(z AS TINYBLOB)",
read={
"mysql": "CAST(x AS MEDIUMBLOB) + CAST(y AS LONGBLOB) + CAST(z AS TINYBLOB)",
},
write={ write={
"mysql": "CAST(x AS CHAR) + CAST(y AS CHAR) + CAST(z AS CHAR)",
"spark": "CAST(x AS BLOB) + CAST(y AS BLOB) + CAST(z AS BLOB)", "spark": "CAST(x AS BLOB) + CAST(y AS BLOB) + CAST(z AS BLOB)",
}, },
) )
self.validate_all("CAST(x AS TIMESTAMP)", write={"mysql": "CAST(x AS DATETIME)"})
self.validate_all("CAST(x AS TIMESTAMPTZ)", write={"mysql": "TIMESTAMP(x)"})
self.validate_all("CAST(x AS TIMESTAMPLTZ)", write={"mysql": "TIMESTAMP(x)"})
def test_canonical_functions(self): def test_canonical_functions(self):
self.validate_identity("SELECT LEFT('str', 2)", "SELECT LEFT('str', 2)") self.validate_identity("SELECT LEFT('str', 2)", "SELECT LEFT('str', 2)")
@ -457,63 +466,63 @@ class TestMySQL(Validator):
"SELECT DATE_FORMAT('2017-06-15', '%Y')", "SELECT DATE_FORMAT('2017-06-15', '%Y')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2017-06-15', '%Y')", "mysql": "SELECT DATE_FORMAT('2017-06-15', '%Y')",
"snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMPNTZ), 'yyyy')", "snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMP), 'yyyy')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2017-06-15', '%m')", "SELECT DATE_FORMAT('2017-06-15', '%m')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2017-06-15', '%m')", "mysql": "SELECT DATE_FORMAT('2017-06-15', '%m')",
"snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMPNTZ), 'mm')", "snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMP), 'mm')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2017-06-15', '%d')", "SELECT DATE_FORMAT('2017-06-15', '%d')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2017-06-15', '%d')", "mysql": "SELECT DATE_FORMAT('2017-06-15', '%d')",
"snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMPNTZ), 'DD')", "snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMP), 'DD')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2017-06-15', '%Y-%m-%d')", "SELECT DATE_FORMAT('2017-06-15', '%Y-%m-%d')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2017-06-15', '%Y-%m-%d')", "mysql": "SELECT DATE_FORMAT('2017-06-15', '%Y-%m-%d')",
"snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMPNTZ), 'yyyy-mm-DD')", "snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMP), 'yyyy-mm-DD')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2017-06-15 22:23:34', '%H')", "SELECT DATE_FORMAT('2017-06-15 22:23:34', '%H')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2017-06-15 22:23:34', '%H')", "mysql": "SELECT DATE_FORMAT('2017-06-15 22:23:34', '%H')",
"snowflake": "SELECT TO_CHAR(CAST('2017-06-15 22:23:34' AS TIMESTAMPNTZ), 'hh24')", "snowflake": "SELECT TO_CHAR(CAST('2017-06-15 22:23:34' AS TIMESTAMP), 'hh24')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2017-06-15', '%w')", "SELECT DATE_FORMAT('2017-06-15', '%w')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2017-06-15', '%w')", "mysql": "SELECT DATE_FORMAT('2017-06-15', '%w')",
"snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMPNTZ), 'dy')", "snowflake": "SELECT TO_CHAR(CAST('2017-06-15' AS TIMESTAMP), 'dy')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2009-10-04 22:23:00', '%W %M %Y')", "SELECT DATE_FORMAT('2009-10-04 22:23:00', '%W %M %Y')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2009-10-04 22:23:00', '%W %M %Y')", "mysql": "SELECT DATE_FORMAT('2009-10-04 22:23:00', '%W %M %Y')",
"snowflake": "SELECT TO_CHAR(CAST('2009-10-04 22:23:00' AS TIMESTAMPNTZ), 'DY mmmm yyyy')", "snowflake": "SELECT TO_CHAR(CAST('2009-10-04 22:23:00' AS TIMESTAMP), 'DY mmmm yyyy')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('2007-10-04 22:23:00', '%H:%i:%s')", "SELECT DATE_FORMAT('2007-10-04 22:23:00', '%H:%i:%s')",
write={ write={
"mysql": "SELECT DATE_FORMAT('2007-10-04 22:23:00', '%T')", "mysql": "SELECT DATE_FORMAT('2007-10-04 22:23:00', '%T')",
"snowflake": "SELECT TO_CHAR(CAST('2007-10-04 22:23:00' AS TIMESTAMPNTZ), 'hh24:mi:ss')", "snowflake": "SELECT TO_CHAR(CAST('2007-10-04 22:23:00' AS TIMESTAMP), 'hh24:mi:ss')",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_FORMAT('1900-10-04 22:23:00', '%d %y %a %d %m %b')", "SELECT DATE_FORMAT('1900-10-04 22:23:00', '%d %y %a %d %m %b')",
write={ write={
"mysql": "SELECT DATE_FORMAT('1900-10-04 22:23:00', '%d %y %W %d %m %b')", "mysql": "SELECT DATE_FORMAT('1900-10-04 22:23:00', '%d %y %W %d %m %b')",
"snowflake": "SELECT TO_CHAR(CAST('1900-10-04 22:23:00' AS TIMESTAMPNTZ), 'DD yy DY DD mm mon')", "snowflake": "SELECT TO_CHAR(CAST('1900-10-04 22:23:00' AS TIMESTAMP), 'DD yy DY DD mm mon')",
}, },
) )
@ -598,6 +607,19 @@ class TestMySQL(Validator):
) )
def test_mysql(self): def test_mysql(self):
self.validate_all(
"SELECT department, GROUP_CONCAT(name) AS employee_names FROM data GROUP BY department",
read={
"postgres": "SELECT department, array_agg(name) AS employee_names FROM data GROUP BY department",
},
)
self.validate_all(
"SELECT UNIX_TIMESTAMP(CAST('2024-04-29 12:00:00' AS DATETIME))",
read={
"mysql": "SELECT UNIX_TIMESTAMP(CAST('2024-04-29 12:00:00' AS DATETIME))",
"postgres": "SELECT EXTRACT(epoch FROM TIMESTAMP '2024-04-29 12:00:00')",
},
)
self.validate_all( self.validate_all(
"SELECT JSON_EXTRACT('[10, 20, [30, 40]]', '$[1]')", "SELECT JSON_EXTRACT('[10, 20, [30, 40]]', '$[1]')",
read={ read={
@ -1109,3 +1131,23 @@ COMMENT='客户账户表'"""
"tsql": "CAST(a AS FLOAT) / NULLIF(b, 0)", "tsql": "CAST(a AS FLOAT) / NULLIF(b, 0)",
}, },
) )
def test_timestamp_trunc(self):
for dialect in ("postgres", "snowflake", "duckdb", "spark", "databricks"):
for unit in (
"MILLISECOND",
"SECOND",
"DAY",
"MONTH",
"YEAR",
):
with self.subTest(f"MySQL -> {dialect} Timestamp Trunc with unit {unit}: "):
self.validate_all(
f"DATE_ADD('0000-01-01 00:00:00', INTERVAL (TIMESTAMPDIFF({unit}, '0000-01-01 00:00:00', CAST('2001-02-16 20:38:40' AS DATETIME))) {unit})",
read={
dialect: f"DATE_TRUNC({unit}, TIMESTAMP '2001-02-16 20:38:40')",
},
write={
"mysql": f"DATE_ADD('0000-01-01 00:00:00', INTERVAL (TIMESTAMPDIFF({unit}, '0000-01-01 00:00:00', CAST('2001-02-16 20:38:40' AS DATETIME))) {unit})",
},
)

39
tests/dialects/test_postgres.py
View file

@ -312,8 +312,32 @@ class TestPostgres(Validator):
"MERGE INTO x USING (SELECT id) AS y ON a = b WHEN MATCHED THEN UPDATE SET x.a = y.b WHEN NOT MATCHED THEN INSERT (a, b) VALUES (y.a, y.b)", "MERGE INTO x USING (SELECT id) AS y ON a = b WHEN MATCHED THEN UPDATE SET x.a = y.b WHEN NOT MATCHED THEN INSERT (a, b) VALUES (y.a, y.b)",
"MERGE INTO x USING (SELECT id) AS y ON a = b WHEN MATCHED THEN UPDATE SET a = y.b WHEN NOT MATCHED THEN INSERT (a, b) VALUES (y.a, y.b)", "MERGE INTO x USING (SELECT id) AS y ON a = b WHEN MATCHED THEN UPDATE SET a = y.b WHEN NOT MATCHED THEN INSERT (a, b) VALUES (y.a, y.b)",
) )
self.validate_identity("SELECT * FROM t1*", "SELECT * FROM t1") self.validate_identity(
"SELECT * FROM t1*",
"SELECT * FROM t1",
)
self.validate_identity(
"SELECT SUBSTRING('afafa' for 1)",
"SELECT SUBSTRING('afafa' FROM 1 FOR 1)",
)
self.validate_identity(
"CAST(x AS INT8)",
"CAST(x AS BIGINT)",
)
self.validate_all(
"SELECT REGEXP_REPLACE('mr .', '[^a-zA-Z]', '', 'g')",
write={
"duckdb": "SELECT REGEXP_REPLACE('mr .', '[^a-zA-Z]', '', 'g')",
"postgres": "SELECT REGEXP_REPLACE('mr .', '[^a-zA-Z]', '', 'g')",
},
)
self.validate_all(
"CREATE TABLE t (c INT)",
read={
"mysql": "CREATE TABLE t (c INT COMMENT 'comment 1') COMMENT = 'comment 2'",
},
)
self.validate_all( self.validate_all(
'SELECT * FROM "test_table" ORDER BY RANDOM() LIMIT 5', 'SELECT * FROM "test_table" ORDER BY RANDOM() LIMIT 5',
write={ write={
@ -449,7 +473,7 @@ class TestPostgres(Validator):
write={ write={
"postgres": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))", "postgres": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))",
"redshift": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))", "redshift": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))",
"snowflake": "SELECT DATE_PART(minute, CAST('2023-01-04 04:05:06.789' AS TIMESTAMPNTZ))", "snowflake": "SELECT DATE_PART(minute, CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))",
}, },
) )
self.validate_all( self.validate_all(
@ -660,6 +684,16 @@ class TestPostgres(Validator):
) )
self.assertIsInstance(self.parse_one("id::UUID"), exp.Cast) self.assertIsInstance(self.parse_one("id::UUID"), exp.Cast)
self.validate_identity(
"COPY tbl (col1, col2) FROM 'file' WITH (FORMAT format, HEADER MATCH, FREEZE TRUE)"
)
self.validate_identity(
"COPY tbl (col1, col2) TO 'file' WITH (FORMAT format, HEADER MATCH, FREEZE TRUE)"
)
self.validate_identity(
"COPY (SELECT * FROM t) TO 'file' WITH (FORMAT format, HEADER MATCH, FREEZE TRUE)"
)
def test_ddl(self): def test_ddl(self):
# Checks that user-defined types are parsed into DataType instead of Identifier # Checks that user-defined types are parsed into DataType instead of Identifier
self.parse_one("CREATE TABLE t (a udt)").this.expressions[0].args["kind"].assert_is( self.parse_one("CREATE TABLE t (a udt)").this.expressions[0].args["kind"].assert_is(
@ -676,6 +710,7 @@ class TestPostgres(Validator):
cdef.args["kind"].assert_is(exp.DataType) cdef.args["kind"].assert_is(exp.DataType)
self.assertEqual(expr.sql(dialect="postgres"), "CREATE TABLE t (x INTERVAL DAY)") self.assertEqual(expr.sql(dialect="postgres"), "CREATE TABLE t (x INTERVAL DAY)")
self.validate_identity("CREATE INDEX IF NOT EXISTS ON t(c)")
self.validate_identity("CREATE INDEX et_vid_idx ON et(vid) INCLUDE (fid)") self.validate_identity("CREATE INDEX et_vid_idx ON et(vid) INCLUDE (fid)")
self.validate_identity("CREATE INDEX idx_x ON x USING BTREE(x, y) WHERE (NOT y IS NULL)") self.validate_identity("CREATE INDEX idx_x ON x USING BTREE(x, y) WHERE (NOT y IS NULL)")
self.validate_identity("CREATE TABLE test (elems JSONB[])") self.validate_identity("CREATE TABLE test (elems JSONB[])")

12
tests/dialects/test_presto.py
View file

@ -10,6 +10,8 @@ class TestPresto(Validator):
self.validate_identity("SELECT * FROM x qualify", "SELECT * FROM x AS qualify") self.validate_identity("SELECT * FROM x qualify", "SELECT * FROM x AS qualify")
self.validate_identity("CAST(x AS IPADDRESS)") self.validate_identity("CAST(x AS IPADDRESS)")
self.validate_identity("CAST(x AS IPPREFIX)") self.validate_identity("CAST(x AS IPPREFIX)")
self.validate_identity("CAST(TDIGEST_AGG(1) AS TDIGEST)")
self.validate_identity("CAST(x AS HYPERLOGLOG)")
self.validate_all( self.validate_all(
"CAST(x AS INTERVAL YEAR TO MONTH)", "CAST(x AS INTERVAL YEAR TO MONTH)",
@ -1059,6 +1061,15 @@ class TestPresto(Validator):
) )
def test_json(self): def test_json(self):
with self.assertLogs(helper_logger):
self.validate_all(
"""SELECT JSON_EXTRACT_SCALAR(TRY(FILTER(CAST(JSON_EXTRACT('{"k1": [{"k2": "{\\"k3\\": 1}", "k4": "v"}]}', '$.k1') AS ARRAY(MAP(VARCHAR, VARCHAR))), x -> x['k4'] = 'v')[1]['k2']), '$.k3')""",
write={
"presto": """SELECT JSON_EXTRACT_SCALAR(TRY(FILTER(CAST(JSON_EXTRACT('{"k1": [{"k2": "{\\"k3\\": 1}", "k4": "v"}]}', '$.k1') AS ARRAY(MAP(VARCHAR, VARCHAR))), x -> x['k4'] = 'v')[1]['k2']), '$.k3')""",
"spark": """SELECT GET_JSON_OBJECT(FILTER(FROM_JSON(GET_JSON_OBJECT('{"k1": [{"k2": "{\\\\"k3\\\\": 1}", "k4": "v"}]}', '$.k1'), 'ARRAY<MAP<STRING, STRING>>'), x -> x['k4'] = 'v')[0]['k2'], '$.k3')""",
},
)
self.validate_all( self.validate_all(
"SELECT CAST(JSON '[1,23,456]' AS ARRAY(INTEGER))", "SELECT CAST(JSON '[1,23,456]' AS ARRAY(INTEGER))",
write={ write={
@ -1073,7 +1084,6 @@ class TestPresto(Validator):
"presto": 'SELECT CAST(JSON_PARSE(\'{"k1":1,"k2":23,"k3":456}\') AS MAP(VARCHAR, INTEGER))', "presto": 'SELECT CAST(JSON_PARSE(\'{"k1":1,"k2":23,"k3":456}\') AS MAP(VARCHAR, INTEGER))',
}, },
) )
self.validate_all( self.validate_all(
"SELECT CAST(ARRAY [1, 23, 456] AS JSON)", "SELECT CAST(ARRAY [1, 23, 456] AS JSON)",
write={ write={

13
tests/dialects/test_prql.py
View file

@ -66,3 +66,16 @@ class TestPRQL(Validator):
"from x filter (a > 1 || null != b || c != null)", "from x filter (a > 1 || null != b || c != null)",
"SELECT * FROM x WHERE (a > 1 OR NOT b IS NULL OR NOT c IS NULL)", "SELECT * FROM x WHERE (a > 1 OR NOT b IS NULL OR NOT c IS NULL)",
) )
self.validate_identity("from a aggregate { average x }", "SELECT AVG(x) FROM a")
self.validate_identity(
"from a aggregate { average x, min y, ct = sum z }",
"SELECT AVG(x), MIN(y), COALESCE(SUM(z), 0) AS ct FROM a",
)
self.validate_identity(
"from a aggregate { average x, min y, sum z }",
"SELECT AVG(x), MIN(y), COALESCE(SUM(z), 0) FROM a",
)
self.validate_identity(
"from a aggregate { min y, b = stddev x, max z }",
"SELECT MIN(y), STDDEV(x) AS b, MAX(z) FROM a",
)

10
tests/dialects/test_redshift.py
View file

@ -162,7 +162,7 @@ class TestRedshift(Validator):
write={ write={
"postgres": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))", "postgres": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))",
"redshift": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))", "redshift": "SELECT EXTRACT(minute FROM CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))",
"snowflake": "SELECT DATE_PART(minute, CAST('2023-01-04 04:05:06.789' AS TIMESTAMPNTZ))", "snowflake": "SELECT DATE_PART(minute, CAST('2023-01-04 04:05:06.789' AS TIMESTAMP))",
}, },
) )
self.validate_all( self.validate_all(
@ -271,7 +271,7 @@ class TestRedshift(Validator):
"postgres": "SELECT CAST('2008-02-28' AS TIMESTAMP) + INTERVAL '18 MONTH'", "postgres": "SELECT CAST('2008-02-28' AS TIMESTAMP) + INTERVAL '18 MONTH'",
"presto": "SELECT DATE_ADD('MONTH', 18, CAST('2008-02-28' AS TIMESTAMP))", "presto": "SELECT DATE_ADD('MONTH', 18, CAST('2008-02-28' AS TIMESTAMP))",
"redshift": "SELECT DATEADD(MONTH, 18, '2008-02-28')", "redshift": "SELECT DATEADD(MONTH, 18, '2008-02-28')",
"snowflake": "SELECT DATEADD(MONTH, 18, CAST('2008-02-28' AS TIMESTAMPNTZ))", "snowflake": "SELECT DATEADD(MONTH, 18, CAST('2008-02-28' AS TIMESTAMP))",
"tsql": "SELECT DATEADD(MONTH, 18, CAST('2008-02-28' AS DATETIME2))", "tsql": "SELECT DATEADD(MONTH, 18, CAST('2008-02-28' AS DATETIME2))",
}, },
) )
@ -362,8 +362,10 @@ class TestRedshift(Validator):
"CREATE TABLE sales (salesid INTEGER NOT NULL) DISTKEY(listid) COMPOUND SORTKEY(listid, sellerid) DISTSTYLE AUTO" "CREATE TABLE sales (salesid INTEGER NOT NULL) DISTKEY(listid) COMPOUND SORTKEY(listid, sellerid) DISTSTYLE AUTO"
) )
self.validate_identity( self.validate_identity(
"COPY customer FROM 's3://mybucket/customer' IAM_ROLE 'arn:aws:iam::0123456789012:role/MyRedshiftRole'", "COPY customer FROM 's3://mybucket/customer' IAM_ROLE 'arn:aws:iam::0123456789012:role/MyRedshiftRole' REGION 'us-east-1' FORMAT orc",
check_command_warning=True, )
self.validate_identity(
"COPY customer FROM 's3://mybucket/mydata' CREDENTIALS 'aws_iam_role=arn:aws:iam::<aws-account-id>:role/<role-name>;master_symmetric_key=<root-key>' emptyasnull blanksasnull timeformat 'YYYY-MM-DD HH:MI:SS'"
) )
self.validate_identity( self.validate_identity(
"UNLOAD ('select * from venue') TO 's3://mybucket/unload/' IAM_ROLE 'arn:aws:iam::0123456789012:role/MyRedshiftRole'", "UNLOAD ('select * from venue') TO 's3://mybucket/unload/' IAM_ROLE 'arn:aws:iam::0123456789012:role/MyRedshiftRole'",

51
tests/dialects/test_snowflake.py
View file

@ -10,6 +10,9 @@ class TestSnowflake(Validator):
dialect = "snowflake" dialect = "snowflake"
def test_snowflake(self): def test_snowflake(self):
self.validate_identity(
"MERGE INTO my_db AS ids USING (SELECT new_id FROM my_model WHERE NOT col IS NULL) AS new_ids ON ids.type = new_ids.type AND ids.source = new_ids.source WHEN NOT MATCHED THEN INSERT VALUES (new_ids.new_id)"
)
self.validate_identity("ALTER TABLE table1 CLUSTER BY (name DESC)") self.validate_identity("ALTER TABLE table1 CLUSTER BY (name DESC)")
self.validate_identity( self.validate_identity(
"INSERT OVERWRITE TABLE t SELECT 1", "INSERT OVERWRITE INTO t SELECT 1" "INSERT OVERWRITE TABLE t SELECT 1", "INSERT OVERWRITE INTO t SELECT 1"
@ -388,7 +391,7 @@ WHERE
"SELECT DATE_PART('year', TIMESTAMP '2020-01-01')", "SELECT DATE_PART('year', TIMESTAMP '2020-01-01')",
write={ write={
"hive": "SELECT EXTRACT(year FROM CAST('2020-01-01' AS TIMESTAMP))", "hive": "SELECT EXTRACT(year FROM CAST('2020-01-01' AS TIMESTAMP))",
"snowflake": "SELECT DATE_PART('year', CAST('2020-01-01' AS TIMESTAMPNTZ))", "snowflake": "SELECT DATE_PART('year', CAST('2020-01-01' AS TIMESTAMP))",
"spark": "SELECT EXTRACT(year FROM CAST('2020-01-01' AS TIMESTAMP))", "spark": "SELECT EXTRACT(year FROM CAST('2020-01-01' AS TIMESTAMP))",
}, },
) )
@ -591,7 +594,7 @@ WHERE
self.validate_all( self.validate_all(
"SELECT DAYOFWEEK('2016-01-02T23:39:20.123-07:00'::TIMESTAMP)", "SELECT DAYOFWEEK('2016-01-02T23:39:20.123-07:00'::TIMESTAMP)",
write={ write={
"snowflake": "SELECT DAYOFWEEK(CAST('2016-01-02T23:39:20.123-07:00' AS TIMESTAMPNTZ))", "snowflake": "SELECT DAYOFWEEK(CAST('2016-01-02T23:39:20.123-07:00' AS TIMESTAMP))",
}, },
) )
self.validate_all( self.validate_all(
@ -689,7 +692,7 @@ WHERE
"SELECT TO_TIMESTAMP('2013-04-05 01:02:03')", "SELECT TO_TIMESTAMP('2013-04-05 01:02:03')",
write={ write={
"bigquery": "SELECT CAST('2013-04-05 01:02:03' AS DATETIME)", "bigquery": "SELECT CAST('2013-04-05 01:02:03' AS DATETIME)",
"snowflake": "SELECT CAST('2013-04-05 01:02:03' AS TIMESTAMPNTZ)", "snowflake": "SELECT CAST('2013-04-05 01:02:03' AS TIMESTAMP)",
"spark": "SELECT CAST('2013-04-05 01:02:03' AS TIMESTAMP)", "spark": "SELECT CAST('2013-04-05 01:02:03' AS TIMESTAMP)",
}, },
) )
@ -878,10 +881,6 @@ WHERE
self.validate_identity("SELECT * FROM @namespace.%table_name/path/to/file.json.gz") self.validate_identity("SELECT * FROM @namespace.%table_name/path/to/file.json.gz")
self.validate_identity("SELECT * FROM '@external/location' (FILE_FORMAT => 'path.to.csv')") self.validate_identity("SELECT * FROM '@external/location' (FILE_FORMAT => 'path.to.csv')")
self.validate_identity("PUT file:///dir/tmp.csv @%table", check_command_warning=True) self.validate_identity("PUT file:///dir/tmp.csv @%table", check_command_warning=True)
self.validate_identity(
'COPY INTO NEW_TABLE ("foo", "bar") FROM (SELECT $1, $2, $3, $4 FROM @%old_table)',
check_command_warning=True,
)
self.validate_identity( self.validate_identity(
"SELECT * FROM @foo/bar (FILE_FORMAT => ds_sandbox.test.my_csv_format, PATTERN => 'test') AS bla" "SELECT * FROM @foo/bar (FILE_FORMAT => ds_sandbox.test.my_csv_format, PATTERN => 'test') AS bla"
) )
@ -955,12 +954,16 @@ WHERE
self.validate_identity("SELECT CAST('12:00:00' AS TIME)") self.validate_identity("SELECT CAST('12:00:00' AS TIME)")
self.validate_identity("SELECT DATE_PART(month, a)") self.validate_identity("SELECT DATE_PART(month, a)")
self.validate_all( for data_type in (
"SELECT CAST(a AS TIMESTAMP)", "TIMESTAMP",
write={ "TIMESTAMPLTZ",
"snowflake": "SELECT CAST(a AS TIMESTAMPNTZ)", "TIMESTAMPNTZ",
}, ):
) self.validate_identity(f"CAST(a AS {data_type})")
self.validate_identity("CAST(a AS TIMESTAMP_NTZ)", "CAST(a AS TIMESTAMPNTZ)")
self.validate_identity("CAST(a AS TIMESTAMP_LTZ)", "CAST(a AS TIMESTAMPLTZ)")
self.validate_all( self.validate_all(
"SELECT a::TIMESTAMP_LTZ(9)", "SELECT a::TIMESTAMP_LTZ(9)",
write={ write={
@ -1000,14 +1003,14 @@ WHERE
self.validate_all( self.validate_all(
"SELECT DATE_PART(epoch_second, foo) as ddate from table_name", "SELECT DATE_PART(epoch_second, foo) as ddate from table_name",
write={ write={
"snowflake": "SELECT EXTRACT(epoch_second FROM CAST(foo AS TIMESTAMPNTZ)) AS ddate FROM table_name", "snowflake": "SELECT EXTRACT(epoch_second FROM CAST(foo AS TIMESTAMP)) AS ddate FROM table_name",
"presto": "SELECT TO_UNIXTIME(CAST(foo AS TIMESTAMP)) AS ddate FROM table_name", "presto": "SELECT TO_UNIXTIME(CAST(foo AS TIMESTAMP)) AS ddate FROM table_name",
}, },
) )
self.validate_all( self.validate_all(
"SELECT DATE_PART(epoch_milliseconds, foo) as ddate from table_name", "SELECT DATE_PART(epoch_milliseconds, foo) as ddate from table_name",
write={ write={
"snowflake": "SELECT EXTRACT(epoch_second FROM CAST(foo AS TIMESTAMPNTZ)) * 1000 AS ddate FROM table_name", "snowflake": "SELECT EXTRACT(epoch_second FROM CAST(foo AS TIMESTAMP)) * 1000 AS ddate FROM table_name",
"presto": "SELECT TO_UNIXTIME(CAST(foo AS TIMESTAMP)) * 1000 AS ddate FROM table_name", "presto": "SELECT TO_UNIXTIME(CAST(foo AS TIMESTAMP)) * 1000 AS ddate FROM table_name",
}, },
) )
@ -1138,7 +1141,7 @@ WHERE
) )
self.validate_identity( self.validate_identity(
"SELECT * FROM my_table AT (TIMESTAMP => 'Fri, 01 May 2015 16:20:00 -0700'::timestamp)", "SELECT * FROM my_table AT (TIMESTAMP => 'Fri, 01 May 2015 16:20:00 -0700'::timestamp)",
"SELECT * FROM my_table AT (TIMESTAMP => CAST('Fri, 01 May 2015 16:20:00 -0700' AS TIMESTAMPNTZ))", "SELECT * FROM my_table AT (TIMESTAMP => CAST('Fri, 01 May 2015 16:20:00 -0700' AS TIMESTAMP))",
) )
self.validate_identity( self.validate_identity(
"SELECT * FROM my_table AT(TIMESTAMP => 'Fri, 01 May 2015 16:20:00 -0700'::timestamp_tz)", "SELECT * FROM my_table AT(TIMESTAMP => 'Fri, 01 May 2015 16:20:00 -0700'::timestamp_tz)",
@ -1581,7 +1584,7 @@ FROM persons AS p, LATERAL FLATTEN(input => p.c, path => 'contact') AS _flattene
"REGEXP_REPLACE(subject, pattern, replacement, position, occurrence, parameters)", "REGEXP_REPLACE(subject, pattern, replacement, position, occurrence, parameters)",
write={ write={
"bigquery": "REGEXP_REPLACE(subject, pattern, replacement)", "bigquery": "REGEXP_REPLACE(subject, pattern, replacement)",
"duckdb": "REGEXP_REPLACE(subject, pattern, replacement)", "duckdb": "REGEXP_REPLACE(subject, pattern, replacement, parameters)",
"hive": "REGEXP_REPLACE(subject, pattern, replacement)", "hive": "REGEXP_REPLACE(subject, pattern, replacement)",
"snowflake": "REGEXP_REPLACE(subject, pattern, replacement, position, occurrence, parameters)", "snowflake": "REGEXP_REPLACE(subject, pattern, replacement, position, occurrence, parameters)",
"spark": "REGEXP_REPLACE(subject, pattern, replacement, position)", "spark": "REGEXP_REPLACE(subject, pattern, replacement, position)",
@ -1827,3 +1830,17 @@ STORAGE_ALLOWED_LOCATIONS=('s3://mybucket1/path1/', 's3://mybucket2/path2/')""",
expression = annotate_types(expression) expression = annotate_types(expression)
self.assertEqual(expression.sql(dialect="snowflake"), "SELECT TRY_CAST(FOO() AS TEXT)") self.assertEqual(expression.sql(dialect="snowflake"), "SELECT TRY_CAST(FOO() AS TEXT)")
def test_copy(self):
self.validate_identity(
"""COPY INTO mytable (col1, col2) FROM 's3://mybucket/data/files' FILES = ('file1', 'file2') PATTERN = 'pattern' FILE_FORMAT = (FORMAT_NAME = my_csv_format NULL_IF = ('str1', 'str2')) PARSE_HEADER = TRUE"""
)
self.validate_identity(
"""COPY INTO temp FROM @random_stage/path/ FILE_FORMAT = (TYPE = CSV FIELD_DELIMITER = '|' NULL_IF = () FIELD_OPTIONALLY_ENCLOSED_BY = '"' TIMESTAMP_FORMAT = 'TZHTZM YYYY-MM-DD HH24:MI:SS.FF9' DATE_FORMAT = 'TZHTZM YYYY-MM-DD HH24:MI:SS.FF9' BINARY_FORMAT = BASE64) VALIDATION_MODE = 'RETURN_3_ROWS'"""
)
self.validate_identity(
"""COPY INTO load1 FROM @%load1/data1/ FILES = ('test1.csv', 'test2.csv') FORCE = TRUE"""
)
self.validate_identity(
"""COPY INTO mytable FROM 'azure://myaccount.blob.core.windows.net/mycontainer/data/files' CREDENTIALS = (AZURE_SAS_TOKEN = 'token') ENCRYPTION = (TYPE = 'AZURE_CSE' MASTER_KEY = 'kPx...') FILE_FORMAT = (FORMAT_NAME = my_csv_format)"""
)

21
tests/dialects/test_spark.py
View file

@ -343,7 +343,7 @@ TBLPROPERTIES (
"postgres": "SELECT CAST('2016-08-31' AS TIMESTAMP) AT TIME ZONE 'Asia/Seoul' AT TIME ZONE 'UTC'", "postgres": "SELECT CAST('2016-08-31' AS TIMESTAMP) AT TIME ZONE 'Asia/Seoul' AT TIME ZONE 'UTC'",
"presto": "SELECT WITH_TIMEZONE(CAST('2016-08-31' AS TIMESTAMP), 'Asia/Seoul') AT TIME ZONE 'UTC'", "presto": "SELECT WITH_TIMEZONE(CAST('2016-08-31' AS TIMESTAMP), 'Asia/Seoul') AT TIME ZONE 'UTC'",
"redshift": "SELECT CAST('2016-08-31' AS TIMESTAMP) AT TIME ZONE 'Asia/Seoul' AT TIME ZONE 'UTC'", "redshift": "SELECT CAST('2016-08-31' AS TIMESTAMP) AT TIME ZONE 'Asia/Seoul' AT TIME ZONE 'UTC'",
"snowflake": "SELECT CONVERT_TIMEZONE('Asia/Seoul', 'UTC', CAST('2016-08-31' AS TIMESTAMPNTZ))", "snowflake": "SELECT CONVERT_TIMEZONE('Asia/Seoul', 'UTC', CAST('2016-08-31' AS TIMESTAMP))",
"spark": "SELECT TO_UTC_TIMESTAMP(CAST('2016-08-31' AS TIMESTAMP), 'Asia/Seoul')", "spark": "SELECT TO_UTC_TIMESTAMP(CAST('2016-08-31' AS TIMESTAMP), 'Asia/Seoul')",
}, },
) )
@ -523,7 +523,14 @@ TBLPROPERTIES (
}, },
) )
for data_type in ("BOOLEAN", "DATE", "DOUBLE", "FLOAT", "INT", "TIMESTAMP"): for data_type in (
"BOOLEAN",
"DATE",
"DOUBLE",
"FLOAT",
"INT",
"TIMESTAMP",
):
self.validate_all( self.validate_all(
f"{data_type}(x)", f"{data_type}(x)",
write={ write={
@ -531,6 +538,16 @@ TBLPROPERTIES (
"spark": f"CAST(x AS {data_type})", "spark": f"CAST(x AS {data_type})",
}, },
) )
for ts_suffix in ("NTZ", "LTZ"):
self.validate_all(
f"TIMESTAMP_{ts_suffix}(x)",
write={
"": f"CAST(x AS TIMESTAMP{ts_suffix})",
"spark": f"CAST(x AS TIMESTAMP_{ts_suffix})",
},
)
self.validate_all( self.validate_all(
"STRING(x)", "STRING(x)",
write={ write={

18
tests/dialects/test_trino.py Normal file
View file

@ -0,0 +1,18 @@
from tests.dialects.test_dialect import Validator
class TestTrino(Validator):
dialect = "trino"
def test_trim(self):
self.validate_identity("SELECT TRIM('!' FROM '!foo!')")
self.validate_identity("SELECT TRIM(BOTH '$' FROM '$var$')")
self.validate_identity("SELECT TRIM(TRAILING 'ER' FROM UPPER('worker'))")
self.validate_identity(
"SELECT TRIM(LEADING FROM ' abcd')",
"SELECT LTRIM(' abcd')",
)
self.validate_identity(
"SELECT TRIM('!foo!', '!')",
"SELECT TRIM('!' FROM '!foo!')",
)

11
tests/dialects/test_tsql.py
View file

@ -29,6 +29,9 @@ class TestTSQL(Validator):
self.validate_identity("1 AND true", "1 <> 0 AND (1 = 1)") self.validate_identity("1 AND true", "1 <> 0 AND (1 = 1)")
self.validate_identity("CAST(x AS int) OR y", "CAST(x AS INTEGER) <> 0 OR y <> 0") self.validate_identity("CAST(x AS int) OR y", "CAST(x AS INTEGER) <> 0 OR y <> 0")
self.validate_identity("TRUNCATE TABLE t1 WITH (PARTITIONS(1, 2 TO 5, 10 TO 20, 84))") self.validate_identity("TRUNCATE TABLE t1 WITH (PARTITIONS(1, 2 TO 5, 10 TO 20, 84))")
self.validate_identity(
"COPY INTO test_1 FROM 'path' WITH (FILE_TYPE = 'CSV', CREDENTIAL = (IDENTITY = 'Shared Access Signature', SECRET = 'token'), FIELDTERMINATOR = ';', ROWTERMINATOR = '0X0A', ENCODING = 'UTF8', DATEFORMAT = 'ymd', MAXERRORS = 10, ERRORFILE = 'errorsfolder', IDENTITY_INSERT = 'ON')"
)
self.validate_all( self.validate_all(
"SELECT IIF(cond <> 0, 'True', 'False')", "SELECT IIF(cond <> 0, 'True', 'False')",
@ -777,6 +780,14 @@ class TestTSQL(Validator):
"CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < CURRENT_TIMESTAMP - 7 END", "CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < CURRENT_TIMESTAMP - 7 END",
"CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < GETDATE() - 7 END", "CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < GETDATE() - 7 END",
) )
self.validate_all(
"CREATE TABLE [#temptest] (name VARCHAR)",
read={
"duckdb": "CREATE TEMPORARY TABLE 'temptest' (name VARCHAR)",
"tsql": "CREATE TABLE [#temptest] (name VARCHAR)",
},
)
self.validate_all( self.validate_all(
"CREATE TABLE tbl (id INTEGER IDENTITY PRIMARY KEY)", "CREATE TABLE tbl (id INTEGER IDENTITY PRIMARY KEY)",
read={ read={

2
tests/fixtures/identity.sql vendored
View file

@ -642,6 +642,7 @@ CREATE TABLE T3 AS (SELECT DISTINCT A FROM T1 EXCEPT (SELECT A FROM T2) LIMIT 1)
DESCRIBE x DESCRIBE x
DESCRIBE EXTENDED a.b DESCRIBE EXTENDED a.b
DESCRIBE FORMATTED a.b DESCRIBE FORMATTED a.b
DESCRIBE SELECT 1
DROP INDEX a.b.c DROP INDEX a.b.c
DROP FUNCTION a.b.c (INT) DROP FUNCTION a.b.c (INT)
DROP MATERIALIZED VIEW x.y.z DROP MATERIALIZED VIEW x.y.z
@ -868,3 +869,4 @@ TRUNCATE(a, b)
SELECT enum SELECT enum
SELECT unlogged SELECT unlogged
SELECT name SELECT name
SELECT copy

4
tests/fixtures/optimizer/qualify_columns.sql vendored
View file

@ -523,6 +523,10 @@ SELECT t.c1 AS c1, t.c3 AS c3 FROM FOO(bar) AS t(c1, c2, c3);
SELECT c.f::VARCHAR(MAX) AS f, e AS e FROM a.b AS c, c.d AS e; SELECT c.f::VARCHAR(MAX) AS f, e AS e FROM a.b AS c, c.d AS e;
SELECT CAST(c.f AS VARCHAR(MAX)) AS f, e AS e FROM a.b AS c, c.d AS e; SELECT CAST(c.f AS VARCHAR(MAX)) AS f, e AS e FROM a.b AS c, c.d AS e;
# dialect: bigquery
WITH cte AS (SELECT 1 AS col) SELECT * FROM cte LEFT JOIN UNNEST((SELECT ARRAY_AGG(DISTINCT x) AS agg FROM UNNEST([1]) AS x WHERE col = 1));
WITH cte AS (SELECT 1 AS col) SELECT * FROM cte AS cte LEFT JOIN UNNEST((SELECT ARRAY_AGG(DISTINCT x) AS agg FROM UNNEST([1]) AS x WHERE cte.col = 1));
-------------------------------------- --------------------------------------
-- Window functions -- Window functions
-------------------------------------- --------------------------------------

14
tests/fixtures/optimizer/qualify_columns_ddl.sql vendored
View file

@ -1,26 +1,26 @@
# title: Create with CTE # title: Create with CTE
WITH cte AS (SELECT b FROM y) CREATE TABLE s AS SELECT * FROM cte; WITH cte AS (SELECT b FROM y) CREATE TABLE s AS SELECT * FROM cte;
CREATE TABLE s AS WITH cte AS (SELECT y.b AS b FROM y AS y) SELECT cte.b AS b FROM cte AS cte; WITH cte AS (SELECT y.b AS b FROM y AS y) CREATE TABLE s AS SELECT cte.b AS b FROM cte AS cte;
# title: Create with CTE, query also has CTE # title: Create with CTE, query also has CTE
WITH cte1 AS (SELECT b FROM y) CREATE TABLE s AS WITH cte2 AS (SELECT b FROM cte1) SELECT * FROM cte2; WITH cte1 AS (SELECT b FROM y) CREATE TABLE s AS WITH cte2 AS (SELECT b FROM cte1) SELECT * FROM cte2;
CREATE TABLE s AS WITH cte1 AS (SELECT y.b AS b FROM y AS y), cte2 AS (SELECT cte1.b AS b FROM cte1 AS cte1) SELECT cte2.b AS b FROM cte2 AS cte2; WITH cte1 AS (SELECT y.b AS b FROM y AS y) CREATE TABLE s AS WITH cte2 AS (SELECT cte1.b AS b FROM cte1 AS cte1) SELECT cte2.b AS b FROM cte2 AS cte2;
# title: Create without CTE # title: Create without CTE
CREATE TABLE foo AS SELECT a FROM tbl; CREATE TABLE foo AS SELECT a FROM tbl;
CREATE TABLE foo AS SELECT tbl.a AS a FROM tbl AS tbl; CREATE TABLE foo AS SELECT tbl.a AS a FROM tbl AS tbl;
# title: Create with complex CTE with derived table # title: Create with complex CTE with derived table
WITH cte AS (SELECT a FROM (SELECT a from x)) CREATE TABLE s AS SELECT * FROM cte; WITH cte AS (SELECT a FROM (SELECT a FROM x)) CREATE TABLE s AS SELECT * FROM cte;
CREATE TABLE s AS WITH cte AS (SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x) AS _q_0) SELECT cte.a AS a FROM cte AS cte; WITH cte AS (SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x) AS _q_0) CREATE TABLE s AS SELECT cte.a AS a FROM cte AS cte;
# title: Create wtih multiple CTEs # title: Create wtih multiple CTEs
WITH cte1 AS (SELECT b FROM y), cte2 AS (SELECT b FROM cte1) CREATE TABLE s AS SELECT * FROM cte2; WITH cte1 AS (SELECT b FROM y), cte2 AS (SELECT b FROM cte1) CREATE TABLE s AS SELECT * FROM cte2;
CREATE TABLE s AS WITH cte1 AS (SELECT y.b AS b FROM y AS y), cte2 AS (SELECT cte1.b AS b FROM cte1 AS cte1) SELECT cte2.b AS b FROM cte2 AS cte2; WITH cte1 AS (SELECT y.b AS b FROM y AS y), cte2 AS (SELECT cte1.b AS b FROM cte1 AS cte1) CREATE TABLE s AS SELECT cte2.b AS b FROM cte2 AS cte2;
# title: Create with multiple CTEs, selecting only from the first CTE (unnecessary code) # title: Create with multiple CTEs, selecting only from the first CTE (unnecessary code)
WITH cte1 AS (SELECT b FROM y), cte2 AS (SELECT b FROM cte1) CREATE TABLE s AS SELECT * FROM cte1; WITH cte1 AS (SELECT b FROM y), cte2 AS (SELECT b FROM cte1) CREATE TABLE s AS SELECT * FROM cte1;
CREATE TABLE s AS WITH cte1 AS (SELECT y.b AS b FROM y AS y), cte2 AS (SELECT cte1.b AS b FROM cte1 AS cte1) SELECT cte1.b AS b FROM cte1 AS cte1; WITH cte1 AS (SELECT y.b AS b FROM y AS y), cte2 AS (SELECT cte1.b AS b FROM cte1 AS cte1) CREATE TABLE s AS SELECT cte1.b AS b FROM cte1 AS cte1;
# title: Create with multiple derived tables # title: Create with multiple derived tables
CREATE TABLE s AS SELECT * FROM (SELECT b FROM (SELECT b FROM y)); CREATE TABLE s AS SELECT * FROM (SELECT b FROM (SELECT b FROM y));
@ -28,7 +28,7 @@ CREATE TABLE s AS SELECT _q_1.b AS b FROM (SELECT _q_0.b AS b FROM (SELECT y.b A
# title: Create with a CTE and a derived table # title: Create with a CTE and a derived table
WITH cte AS (SELECT b FROM y) CREATE TABLE s AS SELECT * FROM (SELECT b FROM (SELECT b FROM cte)); WITH cte AS (SELECT b FROM y) CREATE TABLE s AS SELECT * FROM (SELECT b FROM (SELECT b FROM cte));
CREATE TABLE s AS WITH cte AS (SELECT y.b AS b FROM y AS y) SELECT _q_1.b AS b FROM (SELECT _q_0.b AS b FROM (SELECT cte.b AS b FROM cte AS cte) AS _q_0) AS _q_1; WITH cte AS (SELECT y.b AS b FROM y AS y) CREATE TABLE s AS SELECT _q_1.b AS b FROM (SELECT _q_0.b AS b FROM (SELECT cte.b AS b FROM cte AS cte) AS _q_0) AS _q_1;
# title: Insert with CTE # title: Insert with CTE
# dialect: spark # dialect: spark

4
tests/fixtures/optimizer/qualify_tables.sql vendored
View file

@ -158,6 +158,10 @@ ALTER TABLE c.db.t ADD PRIMARY KEY (id) NOT ENFORCED;
CREATE TABLE t1 AS (WITH cte AS (SELECT x FROM t2) SELECT * FROM cte); CREATE TABLE t1 AS (WITH cte AS (SELECT x FROM t2) SELECT * FROM cte);
CREATE TABLE c.db.t1 AS (WITH cte AS (SELECT x FROM c.db.t2 AS t2) SELECT * FROM cte AS cte); CREATE TABLE c.db.t1 AS (WITH cte AS (SELECT x FROM c.db.t2 AS t2) SELECT * FROM cte AS cte);
# title: delete statement
DELETE FROM t1 WHERE NOT c IN (SELECT c FROM t2);
DELETE FROM c.db.t1 WHERE NOT c IN (SELECT c FROM c.db.t2 AS t2);
# title: insert statement with cte # title: insert statement with cte
# dialect: spark # dialect: spark
WITH cte AS (SELECT b FROM y) INSERT INTO s SELECT * FROM cte; WITH cte AS (SELECT b FROM y) INSERT INTO s SELECT * FROM cte;

11
tests/fixtures/optimizer/simplify.sql vendored
View file

@ -109,6 +109,10 @@ a AND b;
(x is not null) != (y is null); (x is not null) != (y is null);
(NOT x IS NULL) <> (y IS NULL); (NOT x IS NULL) <> (y IS NULL);
# dialect: mysql
A XOR A;
FALSE;
-------------------------------------- --------------------------------------
-- Absorption -- Absorption
-------------------------------------- --------------------------------------
@ -232,6 +236,13 @@ x - 1;
A AND D AND B AND E AND F AND G AND E AND A; A AND D AND B AND E AND F AND G AND E AND A;
A AND B AND D AND E AND F AND G; A AND B AND D AND E AND F AND G;
A OR D OR B OR E OR F OR G OR E OR A;
A OR B OR D OR E OR F OR G;
# dialect: mysql
A XOR D XOR B XOR E XOR F XOR G XOR C;
A XOR B XOR C XOR D XOR E XOR F XOR G;
A AND NOT B AND C AND B; A AND NOT B AND C AND B;
FALSE; FALSE;

4
tests/test_build.py
View file

@ -545,6 +545,10 @@ class TestBuild(unittest.TestCase):
lambda: exp.update("tbl", {"x": 1}, from_="tbl2"), lambda: exp.update("tbl", {"x": 1}, from_="tbl2"),
"UPDATE tbl SET x = 1 FROM tbl2", "UPDATE tbl SET x = 1 FROM tbl2",
), ),
(
lambda: exp.update("tbl", {"x": 1}, from_="tbl2 cross join tbl3"),
"UPDATE tbl SET x = 1 FROM tbl2 CROSS JOIN tbl3",
),
( (
lambda: union("SELECT * FROM foo", "SELECT * FROM bla"), lambda: union("SELECT * FROM foo", "SELECT * FROM bla"),
"SELECT * FROM foo UNION SELECT * FROM bla", "SELECT * FROM foo UNION SELECT * FROM bla",

64
tests/test_lineage.py
View file

@ -224,16 +224,50 @@ class TestLineage(unittest.TestCase):
downstream.source.sql(dialect="snowflake"), downstream.source.sql(dialect="snowflake"),
"LATERAL FLATTEN(INPUT => TEST_TABLE.RESULT, OUTER => TRUE) AS FLATTENED(SEQ, KEY, PATH, INDEX, VALUE, THIS)", "LATERAL FLATTEN(INPUT => TEST_TABLE.RESULT, OUTER => TRUE) AS FLATTENED(SEQ, KEY, PATH, INDEX, VALUE, THIS)",
) )
self.assertEqual( self.assertEqual(downstream.expression.sql(dialect="snowflake"), "VALUE")
downstream.expression.sql(dialect="snowflake"),
"VALUE",
)
self.assertEqual(len(downstream.downstream), 1) self.assertEqual(len(downstream.downstream), 1)
downstream = downstream.downstream[0] downstream = downstream.downstream[0]
self.assertEqual(downstream.name, "TEST_TABLE.RESULT") self.assertEqual(downstream.name, "TEST_TABLE.RESULT")
self.assertEqual(downstream.source.sql(dialect="snowflake"), "TEST_TABLE AS TEST_TABLE") self.assertEqual(downstream.source.sql(dialect="snowflake"), "TEST_TABLE AS TEST_TABLE")
node = lineage(
"FIELD",
"SELECT FLATTENED.VALUE:field::text AS FIELD FROM SNOWFLAKE.SCHEMA.MODEL AS MODEL_ALIAS, LATERAL FLATTEN(INPUT => MODEL_ALIAS.A) AS FLATTENED",
schema={"SNOWFLAKE": {"SCHEMA": {"TABLE": {"A": "integer"}}}},
sources={"SNOWFLAKE.SCHEMA.MODEL": "SELECT A FROM SNOWFLAKE.SCHEMA.TABLE"},
dialect="snowflake",
)
self.assertEqual(node.name, "FIELD")
downstream = node.downstream[0]
self.assertEqual(downstream.name, "FLATTENED.VALUE")
self.assertEqual(
downstream.source.sql(dialect="snowflake"),
"LATERAL FLATTEN(INPUT => MODEL_ALIAS.A) AS FLATTENED(SEQ, KEY, PATH, INDEX, VALUE, THIS)",
)
self.assertEqual(downstream.expression.sql(dialect="snowflake"), "VALUE")
self.assertEqual(len(downstream.downstream), 1)
downstream = downstream.downstream[0]
self.assertEqual(downstream.name, "MODEL_ALIAS.A")
self.assertEqual(downstream.source_name, "SNOWFLAKE.SCHEMA.MODEL")
self.assertEqual(
downstream.source.sql(dialect="snowflake"),
"SELECT TABLE.A AS A FROM SNOWFLAKE.SCHEMA.TABLE AS TABLE",
)
self.assertEqual(downstream.expression.sql(dialect="snowflake"), "TABLE.A AS A")
self.assertEqual(len(downstream.downstream), 1)
downstream = downstream.downstream[0]
self.assertEqual(downstream.name, "TABLE.A")
self.assertEqual(
downstream.source.sql(dialect="snowflake"), "SNOWFLAKE.SCHEMA.TABLE AS TABLE"
)
self.assertEqual(
downstream.expression.sql(dialect="snowflake"), "SNOWFLAKE.SCHEMA.TABLE AS TABLE"
)
def test_subquery(self) -> None: def test_subquery(self) -> None:
node = lineage( node = lineage(
"output", "output",
@ -266,6 +300,7 @@ class TestLineage(unittest.TestCase):
self.assertEqual(node.name, "a") self.assertEqual(node.name, "a")
node = node.downstream[0] node = node.downstream[0]
self.assertEqual(node.name, "cte.a") self.assertEqual(node.name, "cte.a")
self.assertEqual(node.reference_node_name, "cte")
node = node.downstream[0] node = node.downstream[0]
self.assertEqual(node.name, "z.a") self.assertEqual(node.name, "z.a")
@ -304,6 +339,27 @@ class TestLineage(unittest.TestCase):
node = a.downstream[0] node = a.downstream[0]
self.assertEqual(node.name, "foo.a") self.assertEqual(node.name, "foo.a")
# Select from derived table
node = lineage(
"a",
"SELECT a FROM (SELECT a FROM x) subquery",
)
self.assertEqual(node.name, "a")
self.assertEqual(len(node.downstream), 1)
node = node.downstream[0]
self.assertEqual(node.name, "subquery.a")
self.assertEqual(node.reference_node_name, "subquery")
node = lineage(
"a",
"SELECT a FROM (SELECT a FROM x)",
)
self.assertEqual(node.name, "a")
self.assertEqual(len(node.downstream), 1)
node = node.downstream[0]
self.assertEqual(node.name, "_q_0.a")
self.assertEqual(node.reference_node_name, "_q_0")
def test_lineage_cte_union(self) -> None: def test_lineage_cte_union(self) -> None:
query = """ query = """
WITH dataset AS ( WITH dataset AS (