dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Merging upstream version 25.30.0.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-13 21:56:38 +01:00
parent 4816f3663d
commit ebf5336f85
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
69 changed files with 48139 additions and 46098 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

61
CHANGELOG.md
View file

@ -1,6 +1,66 @@
Changelog
=========
## [v25.29.0] - 2024-11-05
### :boom: BREAKING CHANGES
- due to [`e92904e`](https://github.com/tobymao/sqlglot/commit/e92904e61ab3b14fe18d472df19311f9b014f6cc) - Transpile ANY to EXISTS *(PR [#4305](https://github.com/tobymao/sqlglot/pull/4305) by [@VaggelisD](https://github.com/VaggelisD))*:
Transpile ANY to EXISTS (#4305)
- due to [`23e620f`](https://github.com/tobymao/sqlglot/commit/23e620f7cd2860fbce45a5377a75ae0c8f031ce0) - Support MEDIAN() function *(PR [#4317](https://github.com/tobymao/sqlglot/pull/4317) by [@VaggelisD](https://github.com/VaggelisD))*:
Support MEDIAN() function (#4317)
- due to [`a093ae7`](https://github.com/tobymao/sqlglot/commit/a093ae750af8a351e54f1431deba1f2ce6843666) - always wrap value in NOT value IS ... *(PR [#4331](https://github.com/tobymao/sqlglot/pull/4331) by [@georgesittas](https://github.com/georgesittas))*:
always wrap value in NOT value IS ... (#4331)
- due to [`84f78aa`](https://github.com/tobymao/sqlglot/commit/84f78aafd5d7e74da407167cd394d2bff0718cfb) - parse information schema views into a single identifier *(PR [#4336](https://github.com/tobymao/sqlglot/pull/4336) by [@georgesittas](https://github.com/georgesittas))*:
parse information schema views into a single identifier (#4336)
### :sparkles: New Features
- [`efd9b4e`](https://github.com/tobymao/sqlglot/commit/efd9b4ed5a761a2ebfc47a1582e9d1b2eb7cb277) - **postgres**: Support JSONB_EXISTS *(PR [#4302](https://github.com/tobymao/sqlglot/pull/4302) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4299](https://github.com/tobymao/sqlglot/issues/4299) opened by [@dor-bernstein](https://github.com/dor-bernstein)*
- [`e92904e`](https://github.com/tobymao/sqlglot/commit/e92904e61ab3b14fe18d472df19311f9b014f6cc) - **spark**: Transpile ANY to EXISTS *(PR [#4305](https://github.com/tobymao/sqlglot/pull/4305) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4298](https://github.com/tobymao/sqlglot/issues/4298) opened by [@dor-bernstein](https://github.com/dor-bernstein)*
- [`2af4936`](https://github.com/tobymao/sqlglot/commit/2af4936bd9b318c695aae249324ff67bcd1292f6) - **snowflake**: Transpile BQ's TIMESTAMP() function *(PR [#4309](https://github.com/tobymao/sqlglot/pull/4309) by [@VaggelisD](https://github.com/VaggelisD))*
- [`50a1c91`](https://github.com/tobymao/sqlglot/commit/50a1c919d0d46384e3bd9ba1d45c24dd07efe6d2) - **snowflake**: Transpile exp.TimestampAdd *(PR [#4320](https://github.com/tobymao/sqlglot/pull/4320) by [@VaggelisD](https://github.com/VaggelisD))*
- [`01671ce`](https://github.com/tobymao/sqlglot/commit/01671ce137c9cf8d0f12dadc66e0db141f797d16) - **teradata**: add support for hexadecimal literals *(PR [#4323](https://github.com/tobymao/sqlglot/pull/4323) by [@thomascjohnson](https://github.com/thomascjohnson))*
- [`23e620f`](https://github.com/tobymao/sqlglot/commit/23e620f7cd2860fbce45a5377a75ae0c8f031ce0) - Support MEDIAN() function *(PR [#4317](https://github.com/tobymao/sqlglot/pull/4317) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4315](https://github.com/tobymao/sqlglot/issues/4315) opened by [@cpcloud](https://github.com/cpcloud)*
- [`9faef8d`](https://github.com/tobymao/sqlglot/commit/9faef8d1ceff91dd88db46b2c187d64f15490bf4) - **snowflake**: Transpile exp.TimestampSub *(PR [#4329](https://github.com/tobymao/sqlglot/pull/4329) by [@VaggelisD](https://github.com/VaggelisD))*
- [`2d98cac`](https://github.com/tobymao/sqlglot/commit/2d98cacc723bc0c0df8ce11895983fb7cb9f5237) - **BigQuery**: Support JSON_VALUE() *(PR [#4332](https://github.com/tobymao/sqlglot/pull/4332) by [@VaggelisD](https://github.com/VaggelisD))*
- [`f8fec0a`](https://github.com/tobymao/sqlglot/commit/f8fec0ab098df37c3b54d91c24e5d8ec84f7cdbe) - **snowflake**: Transpile exp.DatetimeDiff *(PR [#4334](https://github.com/tobymao/sqlglot/pull/4334) by [@VaggelisD](https://github.com/VaggelisD))*
- [`16fd1ea`](https://github.com/tobymao/sqlglot/commit/16fd1ea2653a602bdc0d8b81e971fb1acadee585) - **BigQuery**: Support JSON_QUERY *(PR [#4333](https://github.com/tobymao/sqlglot/pull/4333) by [@VaggelisD](https://github.com/VaggelisD))*
- [`c09b6a2`](https://github.com/tobymao/sqlglot/commit/c09b6a2a37807795ead251f4fb81a9ba144cce27) - **duckdb**: support flags for RegexpExtract *(PR [#4326](https://github.com/tobymao/sqlglot/pull/4326) by [@NickCrews](https://github.com/NickCrews))*
- [`536973c`](https://github.com/tobymao/sqlglot/commit/536973cfc9d00110e388e8af1ed91d73607e07c2) - **trino**: add support for the ON OVERFLOW clause in LISTAGG *(PR [#4340](https://github.com/tobymao/sqlglot/pull/4340) by [@georgesittas](https://github.com/georgesittas))*
- [`4584935`](https://github.com/tobymao/sqlglot/commit/4584935cab328eced61c62a998cc013cab5cc3e3) - **snowflake**: Transpile exp.StrToDate *(PR [#4348](https://github.com/tobymao/sqlglot/pull/4348) by [@VaggelisD](https://github.com/VaggelisD))*
- [`71f4a47`](https://github.com/tobymao/sqlglot/commit/71f4a47910d5db97fa1a286891d72b5c4694d294) - **snowflake**: Transpile exp.DatetimeAdd *(PR [#4349](https://github.com/tobymao/sqlglot/pull/4349) by [@VaggelisD](https://github.com/VaggelisD))*
### :bug: Bug Fixes
- [`551afff`](https://github.com/tobymao/sqlglot/commit/551afff58ea7bc1047775bfcd5d80b812fb3f682) - handle a Move edge case in the semantic differ *(PR [#4295](https://github.com/tobymao/sqlglot/pull/4295) by [@georgesittas](https://github.com/georgesittas))*
- [`a66e721`](https://github.com/tobymao/sqlglot/commit/a66e721dcd63488f7f3b427569a2115ae044c71b) - **generator**: Add NULL FILTER on ARRAY_AGG only for columns *(PR [#4301](https://github.com/tobymao/sqlglot/pull/4301) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4300](https://github.com/tobymao/sqlglot/issues/4300) opened by [@elad-sachs](https://github.com/elad-sachs)*
- [`b4ea602`](https://github.com/tobymao/sqlglot/commit/b4ea602ab17b0e8e85ddb090156c7bd2c6354de4) - **clickhouse**: improve parsing of WITH FILL ... INTERPOLATE *(PR [#4311](https://github.com/tobymao/sqlglot/pull/4311) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4310](https://github.com/tobymao/sqlglot/issues/4310) opened by [@brunorpinho](https://github.com/brunorpinho)*
- [`749886b`](https://github.com/tobymao/sqlglot/commit/749886b574a5dfa03aeb78b76d9cc097aa0f3e65) - **tsql**: Generate LOG(...) for exp.Ln *(PR [#4318](https://github.com/tobymao/sqlglot/pull/4318) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4316](https://github.com/tobymao/sqlglot/issues/4316) opened by [@cpcloud](https://github.com/cpcloud)*
- [`5c1b1f4`](https://github.com/tobymao/sqlglot/commit/5c1b1f43014967f6853752ba8d0899757a3efcd5) - **parser**: optionally parse a Stream expression *(PR [#4325](https://github.com/tobymao/sqlglot/pull/4325) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4324](https://github.com/tobymao/sqlglot/issues/4324) opened by [@lancewl](https://github.com/lancewl)*
- [`bb49a00`](https://github.com/tobymao/sqlglot/commit/bb49a00b16487356369bbb77aff9c2ff3f9cda52) - **oracle**: Do not normalize time units for exp.DateTrunc *(PR [#4328](https://github.com/tobymao/sqlglot/pull/4328) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4321](https://github.com/tobymao/sqlglot/issues/4321) opened by [@cpcloud](https://github.com/cpcloud)*
- [`a093ae7`](https://github.com/tobymao/sqlglot/commit/a093ae750af8a351e54f1431deba1f2ce6843666) - **clickhouse**: always wrap value in NOT value IS ... *(PR [#4331](https://github.com/tobymao/sqlglot/pull/4331) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4330](https://github.com/tobymao/sqlglot/issues/4330) opened by [@elchyn-cheliabiyeu](https://github.com/elchyn-cheliabiyeu)*
- [`def4f1e`](https://github.com/tobymao/sqlglot/commit/def4f1e3a9eac7545dfad223a5d49cee4fb7eeb8) - Refactor exp.RegexpExtract (follow up 4326) *(PR [#4341](https://github.com/tobymao/sqlglot/pull/4341) by [@VaggelisD](https://github.com/VaggelisD))*
- [`c1456d0`](https://github.com/tobymao/sqlglot/commit/c1456d07097c42a2ba2078ad30a8afe4cc89597d) - presto/trino current_time closes [#4344](https://github.com/tobymao/sqlglot/pull/4344) *(commit by [@tobymao](https://github.com/tobymao))*
- [`8e16abe`](https://github.com/tobymao/sqlglot/commit/8e16abe2fed324b7ed6c718753cc623a8eb37814) - **duckdb**: we ALWAYS need to render group if params is present for RegexpExtract *(PR [#4343](https://github.com/tobymao/sqlglot/pull/4343) by [@NickCrews](https://github.com/NickCrews))*
- [`1689dc7`](https://github.com/tobymao/sqlglot/commit/1689dc7adbb913fe603b5e37eba29cc10d344cd2) - **bigquery**: Parse timezone for DATE_TRUNC *(PR [#4347](https://github.com/tobymao/sqlglot/pull/4347) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4346](https://github.com/tobymao/sqlglot/issues/4346) opened by [@CYFish](https://github.com/CYFish)*
- [`84f78aa`](https://github.com/tobymao/sqlglot/commit/84f78aafd5d7e74da407167cd394d2bff0718cfb) - **bigquery**: parse information schema views into a single identifier *(PR [#4336](https://github.com/tobymao/sqlglot/pull/4336) by [@georgesittas](https://github.com/georgesittas))*
## [v25.28.0] - 2024-10-25
### :boom: BREAKING CHANGES
- due to [`1691388`](https://github.com/tobymao/sqlglot/commit/16913887f5573f01eb8cd2b9336d4b37b84a449a) - Fix chained exp.SetOperation type annotation *(PR [#4274](https://github.com/tobymao/sqlglot/pull/4274) by [@VaggelisD](https://github.com/VaggelisD))*:
@ -5153,3 +5213,4 @@ Changelog
[v25.26.0]: https://github.com/tobymao/sqlglot/compare/v25.25.1...v25.26.0
[v25.27.0]: https://github.com/tobymao/sqlglot/compare/v25.26.0...v25.27.0
[v25.28.0]: https://github.com/tobymao/sqlglot/compare/v25.27.0...v25.28.0
[v25.29.0]: https://github.com/tobymao/sqlglot/compare/v25.28.0...v25.29.0

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

8
docs/sqlglot/_version.html
View file

@ -76,8 +76,8 @@
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;25.28.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">25</span><span class="p">,</span> <span class="mi">28</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;25.29.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">25</span><span class="p">,</span> <span class="mi">29</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div>
@ -97,7 +97,7 @@
<section id="version">
<div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;25.28.0&#39;</span>
<span class="default_value">&#39;25.29.0&#39;</span>
</div>
@ -109,7 +109,7 @@
<section id="version_tuple">
<div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(25, 28, 0)</span>
<span class="default_value">(25, 29, 0)</span>
</div>

9
docs/sqlglot/dialects/athena.html
View file

File diff suppressed because one or more lines are too long

4796
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

5132
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

587
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

86
docs/sqlglot/dialects/dialect.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

3906
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

1699
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/materialize.html
View file

File diff suppressed because one or more lines are too long

3528
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

1943
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

1911
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

2516
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

6
docs/sqlglot/dialects/prql.html
View file

File diff suppressed because one or more lines are too long

1864
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

3
docs/sqlglot/dialects/risingwave.html
View file

@ -1106,6 +1106,8 @@ Default: True</li>
<dd id="RisingWave.Generator.overlay_sql" class="function"><a href="../generator.html#Generator.overlay_sql">overlay_sql</a></dd>
<dd id="RisingWave.Generator.todouble_sql" class="function"><a href="../generator.html#Generator.todouble_sql">todouble_sql</a></dd>
<dd id="RisingWave.Generator.string_sql" class="function"><a href="../generator.html#Generator.string_sql">string_sql</a></dd>
<dd id="RisingWave.Generator.median_sql" class="function"><a href="../generator.html#Generator.median_sql">median_sql</a></dd>
<dd id="RisingWave.Generator.overflowtruncatebehavior_sql" class="function"><a href="../generator.html#Generator.overflowtruncatebehavior_sql">overflowtruncatebehavior_sql</a></dd>
</div>
<div><dt><a href="postgres.html#Postgres.Generator">sqlglot.dialects.postgres.Postgres.Generator</a></dt>
@ -1126,6 +1128,7 @@ Default: True</li>
<dd id="RisingWave.Generator.CAN_IMPLEMENT_ARRAY_ANY" class="variable"><a href="postgres.html#Postgres.Generator.CAN_IMPLEMENT_ARRAY_ANY">CAN_IMPLEMENT_ARRAY_ANY</a></dd>
<dd id="RisingWave.Generator.COPY_HAS_INTO_KEYWORD" class="variable"><a href="postgres.html#Postgres.Generator.COPY_HAS_INTO_KEYWORD">COPY_HAS_INTO_KEYWORD</a></dd>
<dd id="RisingWave.Generator.ARRAY_CONCAT_IS_VAR_LEN" class="variable"><a href="postgres.html#Postgres.Generator.ARRAY_CONCAT_IS_VAR_LEN">ARRAY_CONCAT_IS_VAR_LEN</a></dd>
<dd id="RisingWave.Generator.SUPPORTS_MEDIAN" class="variable"><a href="postgres.html#Postgres.Generator.SUPPORTS_MEDIAN">SUPPORTS_MEDIAN</a></dd>
<dd id="RisingWave.Generator.SUPPORTED_JSON_PATH_PARTS" class="variable"><a href="postgres.html#Postgres.Generator.SUPPORTED_JSON_PATH_PARTS">SUPPORTED_JSON_PATH_PARTS</a></dd>
<dd id="RisingWave.Generator.TYPE_MAPPING" class="variable"><a href="postgres.html#Postgres.Generator.TYPE_MAPPING">TYPE_MAPPING</a></dd>
<dd id="RisingWave.Generator.TRANSFORMS" class="variable"><a href="postgres.html#Postgres.Generator.TRANSFORMS">TRANSFORMS</a></dd>

3262
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

412
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

392
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

1116
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

1770
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

41
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

2640
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

1461
docs/sqlglot/diff.html
View file

File diff suppressed because it is too large Load diff

5
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

12506
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

22168
docs/sqlglot/generator.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/helper.html
View file

@ -1874,7 +1874,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS">
<div class="attr variable">
<span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;year_month&#39;, &#39;quarter&#39;, &#39;week&#39;, &#39;day&#39;, &#39;month&#39;, &#39;year&#39;}</span>
<span class="default_value">{&#39;year_month&#39;, &#39;week&#39;, &#39;month&#39;, &#39;quarter&#39;, &#39;day&#39;, &#39;year&#39;}</span>
</div>

2
docs/sqlglot/jsonpath.html
View file

@ -586,7 +586,7 @@
<div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;}</span>
</div>

4
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -581,7 +581,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;pivots&#39;, &#39;cluster&#39;, &#39;distribute&#39;, &#39;windows&#39;, &#39;locks&#39;, &#39;distinct&#39;, &#39;connect&#39;, &#39;operation_modifiers&#39;, &#39;settings&#39;, &#39;with&#39;, &#39;having&#39;, &#39;qualify&#39;, &#39;limit&#39;, &#39;laterals&#39;, &#39;match&#39;, &#39;options&#39;, &#39;format&#39;, &#39;group&#39;, &#39;sort&#39;, &#39;kind&#39;, &#39;sample&#39;, &#39;offset&#39;, &#39;into&#39;, &#39;prewhere&#39;}</span>
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;having&#39;, &#39;into&#39;, &#39;offset&#39;, &#39;operation_modifiers&#39;, &#39;limit&#39;, &#39;distribute&#39;, &#39;kind&#39;, &#39;qualify&#39;, &#39;settings&#39;, &#39;prewhere&#39;, &#39;sample&#39;, &#39;pivots&#39;, &#39;locks&#39;, &#39;options&#39;, &#39;sort&#39;, &#39;laterals&#39;, &#39;connect&#39;, &#39;with&#39;, &#39;format&#39;, &#39;group&#39;, &#39;cluster&#39;, &#39;windows&#39;, &#39;distinct&#39;, &#39;match&#39;}</span>
</div>

4
docs/sqlglot/optimizer/simplify.html
View file

@ -3231,7 +3231,7 @@ prefix are statically known.</p>
<div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;}</span>
</div>
@ -3315,7 +3315,7 @@ prefix are statically known.</p>
<section id="JOINS">
<div class="attr variable">
<span class="name">JOINS</span> =
<span class="default_value">{(&#39;&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;INNER&#39;)}</span>
<span class="default_value">{(&#39;RIGHT&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;&#39;, &#39;&#39;)}</span>
</div>

17312
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2042
docs/sqlglot/schema.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/tokens.html
View file

@ -9075,7 +9075,7 @@
<div class="attr variable">
<span class="name">COMMANDS</span> =
<input id="Tokenizer.COMMANDS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="Tokenizer.COMMANDS-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.FETCH">TokenType.FETCH</a>: &#39;FETCH&#39;&gt;, &lt;<a href="#TokenType.SHOW">TokenType.SHOW</a>: &#39;SHOW&#39;&gt;, &lt;<a href="#TokenType.RENAME">TokenType.RENAME</a>: &#39;RENAME&#39;&gt;, &lt;<a href="#TokenType.COMMAND">TokenType.COMMAND</a>: &#39;COMMAND&#39;&gt;, &lt;<a href="#TokenType.EXECUTE">TokenType.EXECUTE</a>: &#39;EXECUTE&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="Tokenizer.COMMANDS-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.SHOW">TokenType.SHOW</a>: &#39;SHOW&#39;&gt;, &lt;<a href="#TokenType.FETCH">TokenType.FETCH</a>: &#39;FETCH&#39;&gt;, &lt;<a href="#TokenType.EXECUTE">TokenType.EXECUTE</a>: &#39;EXECUTE&#39;&gt;, &lt;<a href="#TokenType.COMMAND">TokenType.COMMAND</a>: &#39;COMMAND&#39;&gt;, &lt;<a href="#TokenType.RENAME">TokenType.RENAME</a>: &#39;RENAME&#39;&gt;}</span>
</div>

81
docs/sqlglot/transforms.html
View file

@ -105,6 +105,9 @@
<li>
<a class="function" href="#eliminate_join_marks">eliminate_join_marks</a>
</li>
<li>
<a class="function" href="#any_to_exists">any_to_exists</a>
</li>
</ul>
@ -1044,6 +1047,33 @@
</span><span id="L-914"><a href="#L-914"><span class="linenos">914</span></a> <span class="n">where</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="L-915"><a href="#L-915"><span class="linenos">915</span></a>
</span><span id="L-916"><a href="#L-916"><span class="linenos">916</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-917"><a href="#L-917"><span class="linenos">917</span></a>
</span><span id="L-918"><a href="#L-918"><span class="linenos">918</span></a>
</span><span id="L-919"><a href="#L-919"><span class="linenos">919</span></a><span class="k">def</span> <span class="nf">any_to_exists</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-920"><a href="#L-920"><span class="linenos">920</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-921"><a href="#L-921"><span class="linenos">921</span></a><span class="sd"> Transform ANY operator to Spark&#39;s EXISTS</span>
</span><span id="L-922"><a href="#L-922"><span class="linenos">922</span></a>
</span><span id="L-923"><a href="#L-923"><span class="linenos">923</span></a><span class="sd"> For example,</span>
</span><span id="L-924"><a href="#L-924"><span class="linenos">924</span></a><span class="sd"> - Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)</span>
</span><span id="L-925"><a href="#L-925"><span class="linenos">925</span></a><span class="sd"> - Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -&gt; x &lt; 5)</span>
</span><span id="L-926"><a href="#L-926"><span class="linenos">926</span></a>
</span><span id="L-927"><a href="#L-927"><span class="linenos">927</span></a><span class="sd"> Both ANY and EXISTS accept queries but currently only array expressions are supported for this</span>
</span><span id="L-928"><a href="#L-928"><span class="linenos">928</span></a><span class="sd"> transformation</span>
</span><span id="L-929"><a href="#L-929"><span class="linenos">929</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-930"><a href="#L-930"><span class="linenos">930</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-931"><a href="#L-931"><span class="linenos">931</span></a> <span class="k">for</span> <span class="nb">any</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Any</span><span class="p">):</span>
</span><span id="L-932"><a href="#L-932"><span class="linenos">932</span></a> <span class="n">this</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-933"><a href="#L-933"><span class="linenos">933</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="L-934"><a href="#L-934"><span class="linenos">934</span></a> <span class="k">continue</span>
</span><span id="L-935"><a href="#L-935"><span class="linenos">935</span></a>
</span><span id="L-936"><a href="#L-936"><span class="linenos">936</span></a> <span class="n">binop</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">parent</span>
</span><span id="L-937"><a href="#L-937"><span class="linenos">937</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">binop</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span>
</span><span id="L-938"><a href="#L-938"><span class="linenos">938</span></a> <span class="n">lambda_arg</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span>
</span><span id="L-939"><a href="#L-939"><span class="linenos">939</span></a> <span class="nb">any</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">lambda_arg</span><span class="p">)</span>
</span><span id="L-940"><a href="#L-940"><span class="linenos">940</span></a> <span class="n">lambda_expr</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">binop</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">lambda_arg</span><span class="p">])</span>
</span><span id="L-941"><a href="#L-941"><span class="linenos">941</span></a> <span class="n">binop</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Exists</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">this</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">lambda_expr</span><span class="p">))</span>
</span><span id="L-942"><a href="#L-942"><span class="linenos">942</span></a>
</span><span id="L-943"><a href="#L-943"><span class="linenos">943</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -2462,6 +2492,57 @@ If this does not hold for a query, consider running <code><a href="optimizer/qua
</div>
</section>
<section id="any_to_exists">
<input id="any_to_exists-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function">
<span class="def">def</span>
<span class="name">any_to_exists</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="expressions.html#Expression">sqlglot.expressions.Expression</a></span></span><span class="return-annotation">) -> <span class="n"><a href="expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
<label class="view-source-button" for="any_to_exists-view-source"><span>View Source</span></label>
</div>
<a class="headerlink" href="#any_to_exists"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="any_to_exists-920"><a href="#any_to_exists-920"><span class="linenos">920</span></a><span class="k">def</span> <span class="nf">any_to_exists</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="any_to_exists-921"><a href="#any_to_exists-921"><span class="linenos">921</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="any_to_exists-922"><a href="#any_to_exists-922"><span class="linenos">922</span></a><span class="sd"> Transform ANY operator to Spark&#39;s EXISTS</span>
</span><span id="any_to_exists-923"><a href="#any_to_exists-923"><span class="linenos">923</span></a>
</span><span id="any_to_exists-924"><a href="#any_to_exists-924"><span class="linenos">924</span></a><span class="sd"> For example,</span>
</span><span id="any_to_exists-925"><a href="#any_to_exists-925"><span class="linenos">925</span></a><span class="sd"> - Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)</span>
</span><span id="any_to_exists-926"><a href="#any_to_exists-926"><span class="linenos">926</span></a><span class="sd"> - Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -&gt; x &lt; 5)</span>
</span><span id="any_to_exists-927"><a href="#any_to_exists-927"><span class="linenos">927</span></a>
</span><span id="any_to_exists-928"><a href="#any_to_exists-928"><span class="linenos">928</span></a><span class="sd"> Both ANY and EXISTS accept queries but currently only array expressions are supported for this</span>
</span><span id="any_to_exists-929"><a href="#any_to_exists-929"><span class="linenos">929</span></a><span class="sd"> transformation</span>
</span><span id="any_to_exists-930"><a href="#any_to_exists-930"><span class="linenos">930</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="any_to_exists-931"><a href="#any_to_exists-931"><span class="linenos">931</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="any_to_exists-932"><a href="#any_to_exists-932"><span class="linenos">932</span></a> <span class="k">for</span> <span class="nb">any</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Any</span><span class="p">):</span>
</span><span id="any_to_exists-933"><a href="#any_to_exists-933"><span class="linenos">933</span></a> <span class="n">this</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">this</span>
</span><span id="any_to_exists-934"><a href="#any_to_exists-934"><span class="linenos">934</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="any_to_exists-935"><a href="#any_to_exists-935"><span class="linenos">935</span></a> <span class="k">continue</span>
</span><span id="any_to_exists-936"><a href="#any_to_exists-936"><span class="linenos">936</span></a>
</span><span id="any_to_exists-937"><a href="#any_to_exists-937"><span class="linenos">937</span></a> <span class="n">binop</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">parent</span>
</span><span id="any_to_exists-938"><a href="#any_to_exists-938"><span class="linenos">938</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">binop</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span>
</span><span id="any_to_exists-939"><a href="#any_to_exists-939"><span class="linenos">939</span></a> <span class="n">lambda_arg</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span>
</span><span id="any_to_exists-940"><a href="#any_to_exists-940"><span class="linenos">940</span></a> <span class="nb">any</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">lambda_arg</span><span class="p">)</span>
</span><span id="any_to_exists-941"><a href="#any_to_exists-941"><span class="linenos">941</span></a> <span class="n">lambda_expr</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">binop</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">lambda_arg</span><span class="p">])</span>
</span><span id="any_to_exists-942"><a href="#any_to_exists-942"><span class="linenos">942</span></a> <span class="n">binop</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Exists</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">this</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">lambda_expr</span><span class="p">))</span>
</span><span id="any_to_exists-943"><a href="#any_to_exists-943"><span class="linenos">943</span></a>
</span><span id="any_to_exists-944"><a href="#any_to_exists-944"><span class="linenos">944</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
<div class="docstring"><p>Transform ANY operator to Spark's EXISTS</p>
<p>For example,
- Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)
- Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -> x &lt; 5)</p>
<p>Both ANY and EXISTS accept queries but currently only array expressions are supported for this
transformation</p>
</div>
</section>
</main>
<script>

62
sqlglot/dialects/bigquery.py
View file

@ -29,6 +29,7 @@ from sqlglot.dialects.dialect import (
)
from sqlglot.helper import seq_get, split_num_words
from sqlglot.tokens import TokenType
from sqlglot.generator import unsupported_args
if t.TYPE_CHECKING:
from sqlglot._typing import E, Lit
@ -216,26 +217,35 @@ def _build_datetime(args: t.List) -> exp.Func:
return exp.TimestampFromParts.from_arg_list(args)
def _build_regexp_extract(args: t.List) -> exp.RegexpExtract:
def _build_regexp_extract(
expr_type: t.Type[E], default_group: t.Optional[exp.Expression] = None
) -> t.Callable[[t.List], E]:
def _builder(args: t.List) -> E:
try:
group = re.compile(args[1].name).groups == 1
except re.error:
group = False
return exp.RegexpExtract(
# Default group is used for the transpilation of REGEXP_EXTRACT_ALL
return expr_type(
this=seq_get(args, 0),
expression=seq_get(args, 1),
position=seq_get(args, 2),
occurrence=seq_get(args, 3),
group=exp.Literal.number(1) if group else None,
group=exp.Literal.number(1) if group else default_group,
)
return _builder
def _build_json_extract_scalar(args: t.List, dialect: Dialect) -> exp.JSONExtractScalar:
def _build_extract_json_with_default_path(expr_type: t.Type[E]) -> t.Callable[[t.List, Dialect], E]:
def _builder(args: t.List, dialect: Dialect) -> E:
if len(args) == 1:
# The default value for the JSONPath is '$' i.e all of the data
args.append(exp.Literal.string("$"))
return parser.build_extract_json_with_path(exp.JSONExtractScalar)(args, dialect)
return parser.build_extract_json_with_path(expr_type)(args, dialect)
return _builder
def _str_to_datetime_sql(
@ -276,6 +286,24 @@ def _annotate_math_functions(self: TypeAnnotator, expression: E) -> E:
return expression
@unsupported_args("ins_cost", "del_cost", "sub_cost")
def _levenshtein_sql(self: BigQuery.Generator, expression: exp.Levenshtein) -> str:
max_dist = expression.args.get("max_dist")
if max_dist:
max_dist = exp.Kwarg(this=exp.var("max_distance"), expression=max_dist)
return self.func("EDIT_DISTANCE", expression.this, expression.expression, max_dist)
def _build_levenshtein(args: t.List) -> exp.Levenshtein:
max_dist = seq_get(args, 2)
return exp.Levenshtein(
this=seq_get(args, 0),
expression=seq_get(args, 1),
max_dist=max_dist.expression if max_dist else None,
)
class BigQuery(Dialect):
WEEK_OFFSET = -1
UNNEST_COLUMN_ONLY = True
@ -433,16 +461,17 @@ class BigQuery(Dialect):
"DATETIME_ADD": build_date_delta_with_interval(exp.DatetimeAdd),
"DATETIME_SUB": build_date_delta_with_interval(exp.DatetimeSub),
"DIV": binary_from_function(exp.IntDiv),
"EDIT_DISTANCE": lambda args: exp.Levenshtein(
this=seq_get(args, 0), expression=seq_get(args, 1), max_dist=seq_get(args, 2)
),
"EDIT_DISTANCE": _build_levenshtein,
"FORMAT_DATE": lambda args: exp.TimeToStr(
this=exp.TsOrDsToDate(this=seq_get(args, 1)), format=seq_get(args, 0)
),
"GENERATE_ARRAY": exp.GenerateSeries.from_arg_list,
"JSON_EXTRACT_SCALAR": _build_json_extract_scalar,
"JSON_EXTRACT_SCALAR": _build_extract_json_with_default_path(exp.JSONExtractScalar),
"JSON_EXTRACT_ARRAY": _build_extract_json_with_default_path(exp.JSONExtractArray),
"JSON_QUERY": parser.build_extract_json_with_path(exp.JSONExtract),
"JSON_VALUE": _build_json_extract_scalar,
"JSON_QUERY_ARRAY": _build_extract_json_with_default_path(exp.JSONExtractArray),
"JSON_VALUE": _build_extract_json_with_default_path(exp.JSONExtractScalar),
"JSON_VALUE_ARRAY": _build_extract_json_with_default_path(exp.JSONValueArray),
"LENGTH": lambda args: exp.Length(this=seq_get(args, 0), binary=True),
"MD5": exp.MD5Digest.from_arg_list,
"TO_HEX": _build_to_hex,
@ -451,7 +480,11 @@ class BigQuery(Dialect):
),
"PARSE_TIMESTAMP": _build_parse_timestamp,
"REGEXP_CONTAINS": exp.RegexpLike.from_arg_list,
"REGEXP_EXTRACT": _build_regexp_extract,
"REGEXP_EXTRACT": _build_regexp_extract(exp.RegexpExtract),
"REGEXP_SUBSTR": _build_regexp_extract(exp.RegexpExtract),
"REGEXP_EXTRACT_ALL": _build_regexp_extract(
exp.RegexpExtractAll, default_group=exp.Literal.number(0)
),
"SHA256": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(256)),
"SHA512": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(512)),
"SPLIT": lambda args: exp.Split(
@ -735,6 +768,7 @@ class BigQuery(Dialect):
WITH_PROPERTIES_PREFIX = "OPTIONS"
SUPPORTS_EXPLODING_PROJECTIONS = False
EXCEPT_INTERSECT_SUPPORT_ALL_CLAUSE = False
SUPPORTS_UNIX_SECONDS = True
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
@ -744,7 +778,6 @@ class BigQuery(Dialect):
exp.Array: inline_array_unless_query,
exp.ArrayContains: _array_contains_sql,
exp.ArrayFilter: filter_array_using_unnest,
exp.ArraySize: rename_func("ARRAY_LENGTH"),
exp.Cast: transforms.preprocess([transforms.remove_precision_parameterized_types]),
exp.CollateProperty: lambda self, e: (
f"DEFAULT COLLATE {self.sql(e, 'this')}"
@ -777,7 +810,7 @@ class BigQuery(Dialect):
exp.ILike: no_ilike_sql,
exp.IntDiv: rename_func("DIV"),
exp.JSONFormat: rename_func("TO_JSON_STRING"),
exp.Levenshtein: rename_func("EDIT_DISTANCE"),
exp.Levenshtein: _levenshtein_sql,
exp.Max: max_or_greatest,
exp.MD5: lambda self, e: self.func("TO_HEX", self.func("MD5", e.this)),
exp.MD5Digest: rename_func("MD5"),
@ -790,6 +823,9 @@ class BigQuery(Dialect):
e.args.get("position"),
e.args.get("occurrence"),
),
exp.RegexpExtractAll: lambda self, e: self.func(
"REGEXP_EXTRACT_ALL", e.this, e.expression
),
exp.RegexpReplace: regexp_replace_sql,
exp.RegexpLike: rename_func("REGEXP_CONTAINS"),
exp.ReturnsProperty: _returnsproperty_sql,

9
sqlglot/dialects/clickhouse.py
View file

@ -23,6 +23,7 @@ from sqlglot.dialects.dialect import (
from sqlglot.generator import Generator
from sqlglot.helper import is_int, seq_get
from sqlglot.tokens import Token, TokenType
from sqlglot.generator import unsupported_args
DATEΤΙΜΕ_DELTA = t.Union[exp.DateAdd, exp.DateDiff, exp.DateSub, exp.TimestampSub, exp.TimestampAdd]
@ -271,6 +272,8 @@ class ClickHouse(Dialect):
"MD5": exp.MD5Digest.from_arg_list,
"SHA256": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(256)),
"SHA512": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(512)),
"EDITDISTANCE": exp.Levenshtein.from_arg_list,
"LEVENSHTEINDISTANCE": exp.Levenshtein.from_arg_list,
}
AGG_FUNCTIONS = {
@ -850,6 +853,7 @@ class ClickHouse(Dialect):
SET_OP_MODIFIERS = False
SUPPORTS_TABLE_ALIAS_COLUMNS = False
VALUES_AS_TABLE = False
ARRAY_SIZE_NAME = "LENGTH"
STRING_TYPE_MAPPING = {
exp.DataType.Type.CHAR: "String",
@ -925,7 +929,6 @@ class ClickHouse(Dialect):
exp.AnyValue: rename_func("any"),
exp.ApproxDistinct: rename_func("uniq"),
exp.ArrayFilter: lambda self, e: self.func("arrayFilter", e.expression, e.this),
exp.ArraySize: rename_func("LENGTH"),
exp.ArraySum: rename_func("arraySum"),
exp.ArgMax: arg_max_or_min_no_count("argMax"),
exp.ArgMin: arg_max_or_min_no_count("argMin"),
@ -949,6 +952,7 @@ class ClickHouse(Dialect):
exp.JSONPathKey: json_path_key_only_name,
exp.JSONPathRoot: lambda *_: "",
exp.Map: lambda self, e: _lower_func(var_map_sql(self, e)),
exp.Median: rename_func("median"),
exp.Nullif: rename_func("nullIf"),
exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}",
exp.Pivot: no_pivot_sql,
@ -984,6 +988,9 @@ class ClickHouse(Dialect):
exp.Lead: lambda self, e: self.func(
"leadInFrame", e.this, e.args.get("offset"), e.args.get("default")
),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost", "max_dist")(
rename_func("editDistance")
),
}
PROPERTIES_LOCATION = {

4
sqlglot/dialects/databricks.py
View file

@ -7,7 +7,6 @@ from sqlglot.dialects.dialect import (
date_delta_sql,
build_date_delta,
timestamptrunc_sql,
timestampdiff_sql,
)
from sqlglot.dialects.spark import Spark
from sqlglot.tokens import TokenType
@ -46,7 +45,6 @@ class Databricks(Spark):
"DATE_ADD": build_date_delta(exp.DateAdd),
"DATEDIFF": build_date_delta(exp.DateDiff),
"DATE_DIFF": build_date_delta(exp.DateDiff),
"TIMESTAMPDIFF": build_date_delta(exp.TimestampDiff),
"GET_JSON_OBJECT": _build_json_extract,
}
@ -75,8 +73,6 @@ class Databricks(Spark):
exp.Mul(this=e.expression, expression=exp.Literal.number(-1)),
e.this,
),
exp.DatetimeDiff: timestampdiff_sql,
exp.TimestampDiff: timestampdiff_sql,
exp.DatetimeTrunc: timestamptrunc_sql(),
exp.Select: transforms.preprocess(
[

13
sqlglot/dialects/dialect.py
View file

@ -1364,14 +1364,16 @@ def concat_ws_to_dpipe_sql(self: Generator, expression: exp.ConcatWs) -> str:
@unsupported_args("position", "occurrence", "parameters")
def regexp_extract_sql(self: Generator, expression: exp.RegexpExtract) -> str:
def regexp_extract_sql(
self: Generator, expression: exp.RegexpExtract | exp.RegexpExtractAll
) -> str:
group = expression.args.get("group")
# Do not render group if it's the default value for this dialect
if group and group.name == str(self.dialect.REGEXP_EXTRACT_DEFAULT_GROUP):
group = None
return self.func("REGEXP_EXTRACT", expression.this, expression.expression, group)
return self.func(expression.sql_name(), expression.this, expression.expression, group)
@unsupported_args("position", "occurrence", "modifiers")
@ -1693,14 +1695,17 @@ def sequence_sql(self: Generator, expression: exp.GenerateSeries | exp.GenerateD
return self.func("SEQUENCE", start, end, step)
def build_regexp_extract(args: t.List, dialect: Dialect) -> exp.RegexpExtract:
return exp.RegexpExtract(
def build_regexp_extract(expr_type: t.Type[E]) -> t.Callable[[t.List, Dialect], E]:
def _builder(args: t.List, dialect: Dialect) -> E:
return expr_type(
this=seq_get(args, 0),
expression=seq_get(args, 1),
group=seq_get(args, 2) or exp.Literal.number(dialect.REGEXP_EXTRACT_DEFAULT_GROUP),
parameters=seq_get(args, 3),
)
return _builder
def explode_to_unnest_sql(self: Generator, expression: exp.Lateral) -> str:
if isinstance(expression.this, exp.Explode):

9
sqlglot/dialects/drill.py
View file

@ -13,6 +13,7 @@ from sqlglot.dialects.dialect import (
)
from sqlglot.dialects.mysql import date_add_sql
from sqlglot.transforms import preprocess, move_schema_columns_to_partitioned_by
from sqlglot.generator import unsupported_args
def _str_to_date(self: Drill.Generator, expression: exp.StrToDate) -> str:
@ -78,8 +79,10 @@ class Drill(Dialect):
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"REPEATED_COUNT": exp.ArraySize.from_arg_list,
"TO_TIMESTAMP": exp.TimeStrToTime.from_arg_list,
"TO_CHAR": build_formatted_time(exp.TimeToStr, "drill"),
"LEVENSHTEIN_DISTANCE": exp.Levenshtein.from_arg_list,
}
LOG_DEFAULTS_TO_LN = True
@ -91,6 +94,7 @@ class Drill(Dialect):
NVL2_SUPPORTED = False
LAST_DAY_SUPPORTS_DATE_PART = False
SUPPORTS_CREATE_TABLE_LIKE = False
ARRAY_SIZE_NAME = "REPEATED_COUNT"
TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING,
@ -115,7 +119,6 @@ class Drill(Dialect):
**generator.Generator.TRANSFORMS,
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
exp.ArrayContains: rename_func("REPEATED_CONTAINS"),
exp.ArraySize: rename_func("REPEATED_COUNT"),
exp.Create: preprocess([move_schema_columns_to_partitioned_by]),
exp.DateAdd: date_add_sql("ADD"),
exp.DateStrToDate: datestrtodate_sql,
@ -127,7 +130,9 @@ class Drill(Dialect):
exp.If: lambda self,
e: f"`IF`({self.format_args(e.this, e.args.get('true'), e.args.get('false'))})",
exp.ILike: lambda self, e: self.binary(e, "`ILIKE`"),
exp.Levenshtein: rename_func("LEVENSHTEIN_DISTANCE"),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost", "max_dist")(
rename_func("LEVENSHTEIN_DISTANCE")
),
exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}",
exp.RegexpLike: rename_func("REGEXP_MATCHES"),
exp.StrPosition: str_position_sql,

33
sqlglot/dialects/duckdb.py
View file

@ -156,18 +156,24 @@ def _struct_sql(self: DuckDB.Generator, expression: exp.Struct) -> str:
# BigQuery allows inline construction such as "STRUCT<a STRING, b INTEGER>('str', 1)" which is
# canonicalized to "ROW('str', 1) AS STRUCT(a TEXT, b INT)" in DuckDB
# The transformation to ROW will take place if a cast to STRUCT / ARRAY of STRUCTs is found
# The transformation to ROW will take place if:
# 1. The STRUCT itself does not have proper fields (key := value) as a "proper" STRUCT would
# 2. A cast to STRUCT / ARRAY of STRUCTs is found
ancestor_cast = expression.find_ancestor(exp.Cast)
is_struct_cast = ancestor_cast and any(
is_bq_inline_struct = (
(expression.find(exp.PropertyEQ) is None)
and ancestor_cast
and any(
casted_type.is_type(exp.DataType.Type.STRUCT)
for casted_type in ancestor_cast.find_all(exp.DataType)
)
)
for i, expr in enumerate(expression.expressions):
is_property_eq = isinstance(expr, exp.PropertyEQ)
value = expr.expression if is_property_eq else expr
if is_struct_cast:
if is_bq_inline_struct:
args.append(self.sql(value))
else:
key = expr.name if is_property_eq else f"_{i}"
@ -175,7 +181,7 @@ def _struct_sql(self: DuckDB.Generator, expression: exp.Struct) -> str:
csv_args = ", ".join(args)
return f"ROW({csv_args})" if is_struct_cast else f"{{{csv_args}}}"
return f"ROW({csv_args})" if is_bq_inline_struct else f"{{{csv_args}}}"
def _datatype_sql(self: DuckDB.Generator, expression: exp.DataType) -> str:
@ -257,6 +263,14 @@ def _generate_datetime_array_sql(
return self.sql(gen_series)
def _json_extract_value_array_sql(
self: DuckDB.Generator, expression: exp.JSONValueArray | exp.JSONExtractArray
) -> str:
json_extract = exp.JSONExtract(this=expression.this, expression=expression.expression)
data_type = "ARRAY<STRING>" if isinstance(expression, exp.JSONValueArray) else "ARRAY<JSON>"
return self.sql(exp.cast(json_extract, to=exp.DataType.build(data_type)))
class DuckDB(Dialect):
NULL_ORDERING = "nulls_are_last"
SUPPORTS_USER_DEFINED_TYPES = False
@ -376,7 +390,8 @@ class DuckDB(Dialect):
"MAKE_TIMESTAMP": _build_make_timestamp,
"QUANTILE_CONT": exp.PercentileCont.from_arg_list,
"QUANTILE_DISC": exp.PercentileDisc.from_arg_list,
"REGEXP_EXTRACT": build_regexp_extract,
"REGEXP_EXTRACT": build_regexp_extract(exp.RegexpExtract),
"REGEXP_EXTRACT_ALL": build_regexp_extract(exp.RegexpExtractAll),
"REGEXP_MATCHES": exp.RegexpLike.from_arg_list,
"REGEXP_REPLACE": lambda args: exp.RegexpReplace(
this=seq_get(args, 0),
@ -397,6 +412,7 @@ class DuckDB(Dialect):
"XOR": binary_from_function(exp.BitwiseXor),
"GENERATE_SERIES": _build_generate_series(),
"RANGE": _build_generate_series(end_exclusive=True),
"EDITDIST3": exp.Levenshtein.from_arg_list,
}
FUNCTIONS.pop("DATE_SUB")
@ -491,13 +507,13 @@ class DuckDB(Dialect):
STAR_EXCEPT = "EXCLUDE"
PAD_FILL_PATTERN_IS_REQUIRED = True
ARRAY_CONCAT_IS_VAR_LEN = False
ARRAY_SIZE_DIM_REQUIRED = False
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
exp.ApproxDistinct: approx_count_distinct_sql,
exp.Array: inline_array_unless_query,
exp.ArrayFilter: rename_func("LIST_FILTER"),
exp.ArraySize: rename_func("ARRAY_LENGTH"),
exp.ArgMax: arg_max_or_min_no_count("ARG_MAX"),
exp.ArgMin: arg_max_or_min_no_count("ARG_MIN"),
exp.ArraySort: _array_sort_sql,
@ -535,8 +551,10 @@ class DuckDB(Dialect):
exp.IsNan: rename_func("ISNAN"),
exp.JSONBExists: rename_func("JSON_EXISTS"),
exp.JSONExtract: _arrow_json_extract_sql,
exp.JSONExtractArray: _json_extract_value_array_sql,
exp.JSONExtractScalar: _arrow_json_extract_sql,
exp.JSONFormat: _json_format_sql,
exp.JSONValueArray: _json_extract_value_array_sql,
exp.Lateral: explode_to_unnest_sql,
exp.LogicalOr: rename_func("BOOL_OR"),
exp.LogicalAnd: rename_func("BOOL_AND"),
@ -610,6 +628,9 @@ class DuckDB(Dialect):
exp.VariancePop: rename_func("VAR_POP"),
exp.WeekOfYear: rename_func("WEEKOFYEAR"),
exp.Xor: bool_xor_sql,
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost", "max_dist")(
rename_func("LEVENSHTEIN")
),
}
SUPPORTED_JSON_PATH_PARTS = {

10
sqlglot/dialects/hive.py
View file

@ -44,6 +44,7 @@ from sqlglot.transforms import (
)
from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType
from sqlglot.generator import unsupported_args
# (FuncType, Multiplier)
DATE_DELTA_INTERVAL = {
@ -309,7 +310,8 @@ class Hive(Dialect):
"MONTH": lambda args: exp.Month(this=exp.TsOrDsToDate.from_arg_list(args)),
"PERCENTILE": exp.Quantile.from_arg_list,
"PERCENTILE_APPROX": exp.ApproxQuantile.from_arg_list,
"REGEXP_EXTRACT": build_regexp_extract,
"REGEXP_EXTRACT": build_regexp_extract(exp.RegexpExtract),
"REGEXP_EXTRACT_ALL": build_regexp_extract(exp.RegexpExtractAll),
"SEQUENCE": exp.GenerateSeries.from_arg_list,
"SIZE": exp.ArraySize.from_arg_list,
"SPLIT": exp.RegexpSplit.from_arg_list,
@ -461,6 +463,7 @@ class Hive(Dialect):
PARSE_JSON_NAME = None
PAD_FILL_PATTERN_IS_REQUIRED = True
SUPPORTS_MEDIAN = False
ARRAY_SIZE_NAME = "SIZE"
EXPRESSIONS_WITHOUT_NESTED_CTES = {
exp.Insert,
@ -498,7 +501,6 @@ class Hive(Dialect):
exp.ArgMin: arg_max_or_min_no_count("MIN_BY"),
exp.ArrayConcat: rename_func("CONCAT"),
exp.ArrayToString: lambda self, e: self.func("CONCAT_WS", e.expression, e.this),
exp.ArraySize: rename_func("SIZE"),
exp.ArraySort: _array_sort_sql,
exp.With: no_recursive_cte_sql,
exp.DateAdd: _add_date_sql,
@ -542,6 +544,7 @@ class Hive(Dialect):
exp.Quantile: rename_func("PERCENTILE"),
exp.ApproxQuantile: rename_func("PERCENTILE_APPROX"),
exp.RegexpExtract: regexp_extract_sql,
exp.RegexpExtractAll: regexp_extract_sql,
exp.RegexpReplace: regexp_replace_sql,
exp.RegexpLike: lambda self, e: self.binary(e, "RLIKE"),
exp.RegexpSplit: rename_func("SPLIT"),
@ -598,6 +601,9 @@ class Hive(Dialect):
exp.WeekOfYear: rename_func("WEEKOFYEAR"),
exp.DayOfMonth: rename_func("DAYOFMONTH"),
exp.DayOfWeek: rename_func("DAYOFWEEK"),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost", "max_dist")(
rename_func("LEVENSHTEIN")
),
}
PROPERTIES_LOCATION = {

27
sqlglot/dialects/postgres.py
View file

@ -237,6 +237,29 @@ def _unix_to_time_sql(self: Postgres.Generator, expression: exp.UnixToTime) -> s
)
def _build_levenshtein_less_equal(args: t.List) -> exp.Levenshtein:
# Postgres has two signatures for levenshtein_less_equal function, but in both cases
# max_dist is the last argument
# levenshtein_less_equal(source, target, ins_cost, del_cost, sub_cost, max_d)
# levenshtein_less_equal(source, target, max_d)
max_dist = args.pop()
return exp.Levenshtein(
this=seq_get(args, 0),
expression=seq_get(args, 1),
ins_cost=seq_get(args, 2),
del_cost=seq_get(args, 3),
sub_cost=seq_get(args, 4),
max_dist=max_dist,
)
def _levenshtein_sql(self: Postgres.Generator, expression: exp.Levenshtein) -> str:
name = "LEVENSHTEIN_LESS_EQUAL" if expression.args.get("max_dist") else "LEVENSHTEIN"
return rename_func(name)(self, expression)
class Postgres(Dialect):
INDEX_OFFSET = 1
TYPED_DIVISION = True
@ -365,6 +388,7 @@ class Postgres(Dialect):
"SHA256": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(256)),
"SHA384": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(384)),
"SHA512": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(512)),
"LEVENSHTEIN_LESS_EQUAL": _build_levenshtein_less_equal,
}
FUNCTION_PARSERS = {
@ -472,6 +496,7 @@ class Postgres(Dialect):
COPY_HAS_INTO_KEYWORD = False
ARRAY_CONCAT_IS_VAR_LEN = False
SUPPORTS_MEDIAN = False
ARRAY_SIZE_DIM_REQUIRED = True
SUPPORTED_JSON_PATH_PARTS = {
exp.JSONPathKey,
@ -495,7 +520,6 @@ class Postgres(Dialect):
exp.AnyValue: any_value_to_max_sql,
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CAT"),
exp.ArrayFilter: filter_array_using_unnest,
exp.ArraySize: lambda self, e: self.func("ARRAY_LENGTH", e.this, e.expression or "1"),
exp.BitwiseXor: lambda self, e: self.binary(e, "#"),
exp.ColumnDef: transforms.preprocess([_auto_increment_to_serial, _serial_to_generated]),
exp.CurrentDate: no_paren_current_date_sql,
@ -568,6 +592,7 @@ class Postgres(Dialect):
exp.Variance: rename_func("VAR_SAMP"),
exp.Xor: bool_xor_sql,
exp.UnixToTime: _unix_to_time_sql,
exp.Levenshtein: _levenshtein_sql,
}
TRANSFORMS.pop(exp.CommentColumnConstraint)

12
sqlglot/dialects/presto.py
View file

@ -37,6 +37,7 @@ from sqlglot.dialects.mysql import MySQL
from sqlglot.helper import apply_index_offset, seq_get
from sqlglot.tokens import TokenType
from sqlglot.transforms import unqualify_columns
from sqlglot.generator import unsupported_args
DATE_ADD_OR_SUB = t.Union[exp.DateAdd, exp.TimestampAdd, exp.DateSub]
@ -272,8 +273,10 @@ class Presto(Dialect):
"FROM_UTF8": lambda args: exp.Decode(
this=seq_get(args, 0), replace=seq_get(args, 1), charset=exp.Literal.string("utf-8")
),
"LEVENSHTEIN_DISTANCE": exp.Levenshtein.from_arg_list,
"NOW": exp.CurrentTimestamp.from_arg_list,
"REGEXP_EXTRACT": build_regexp_extract,
"REGEXP_EXTRACT": build_regexp_extract(exp.RegexpExtract),
"REGEXP_EXTRACT_ALL": build_regexp_extract(exp.RegexpExtractAll),
"REGEXP_REPLACE": lambda args: exp.RegexpReplace(
this=seq_get(args, 0),
expression=seq_get(args, 1),
@ -318,6 +321,7 @@ class Presto(Dialect):
PAD_FILL_PATTERN_IS_REQUIRED = True
EXCEPT_INTERSECT_SUPPORT_ALL_CLAUSE = False
SUPPORTS_MEDIAN = False
ARRAY_SIZE_NAME = "CARDINALITY"
PROPERTIES_LOCATION = {
**generator.Generator.PROPERTIES_LOCATION,
@ -351,7 +355,6 @@ class Presto(Dialect):
exp.ArrayAny: rename_func("ANY_MATCH"),
exp.ArrayConcat: rename_func("CONCAT"),
exp.ArrayContains: rename_func("CONTAINS"),
exp.ArraySize: rename_func("CARDINALITY"),
exp.ArrayToString: rename_func("ARRAY_JOIN"),
exp.ArrayUniqueAgg: rename_func("SET_AGG"),
exp.AtTimeZone: rename_func("AT_TIMEZONE"),
@ -399,12 +402,15 @@ class Presto(Dialect):
exp.LastDay: lambda self, e: self.func("LAST_DAY_OF_MONTH", e.this),
exp.Lateral: explode_to_unnest_sql,
exp.Left: left_to_substring_sql,
exp.Levenshtein: rename_func("LEVENSHTEIN_DISTANCE"),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost", "max_dist")(
rename_func("LEVENSHTEIN_DISTANCE")
),
exp.LogicalAnd: rename_func("BOOL_AND"),
exp.LogicalOr: rename_func("BOOL_OR"),
exp.Pivot: no_pivot_sql,
exp.Quantile: _quantile_sql,
exp.RegexpExtract: regexp_extract_sql,
exp.RegexpExtractAll: regexp_extract_sql,
exp.Right: right_to_substring_sql,
exp.SafeDivide: no_safe_divide_sql,
exp.Schema: _schema_sql,

101
sqlglot/dialects/snowflake.py
View file

@ -235,6 +235,61 @@ def _unnest_generate_date_array(expression: exp.Expression) -> exp.Expression:
return expression
def _build_regexp_extract(expr_type: t.Type[E]) -> t.Callable[[t.List], E]:
def _builder(args: t.List) -> E:
return expr_type(
this=seq_get(args, 0),
expression=seq_get(args, 1),
position=seq_get(args, 2),
occurrence=seq_get(args, 3),
parameters=seq_get(args, 4),
group=seq_get(args, 5) or exp.Literal.number(0),
)
return _builder
def _regexpextract_sql(self, expression: exp.RegexpExtract | exp.RegexpExtractAll) -> str:
# Other dialects don't support all of the following parameters, so we need to
# generate default values as necessary to ensure the transpilation is correct
group = expression.args.get("group")
# To avoid generating all these default values, we set group to None if
# it's 0 (also default value) which doesn't trigger the following chain
if group and group.name == "0":
group = None
parameters = expression.args.get("parameters") or (group and exp.Literal.string("c"))
occurrence = expression.args.get("occurrence") or (parameters and exp.Literal.number(1))
position = expression.args.get("position") or (occurrence and exp.Literal.number(1))
return self.func(
"REGEXP_SUBSTR" if isinstance(expression, exp.RegexpExtract) else "REGEXP_EXTRACT_ALL",
expression.this,
expression.expression,
position,
occurrence,
parameters,
group,
)
def _json_extract_value_array_sql(
self: Snowflake.Generator, expression: exp.JSONValueArray | exp.JSONExtractArray
) -> str:
json_extract = exp.JSONExtract(this=expression.this, expression=expression.expression)
ident = exp.to_identifier("x")
if isinstance(expression, exp.JSONValueArray):
this: exp.Expression = exp.cast(ident, to=exp.DataType.Type.VARCHAR)
else:
this = exp.ParseJSON(this=f"TO_JSON({ident})")
transform_lambda = exp.Lambda(expressions=[ident], this=this)
return self.func("TRANSFORM", json_extract, transform_lambda)
class Snowflake(Dialect):
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax
NORMALIZATION_STRATEGY = NormalizationStrategy.UPPERCASE
@ -322,6 +377,9 @@ class Snowflake(Dialect):
"DATEADD": _build_date_time_add(exp.DateAdd),
"DATEDIFF": _build_datediff,
"DIV0": _build_if_from_div0,
"EDITDISTANCE": lambda args: exp.Levenshtein(
this=seq_get(args, 0), expression=seq_get(args, 1), max_dist=seq_get(args, 2)
),
"FLATTEN": exp.Explode.from_arg_list,
"GET_PATH": lambda args, dialect: exp.JSONExtract(
this=seq_get(args, 0), expression=dialect.to_json_path(seq_get(args, 1))
@ -335,15 +393,10 @@ class Snowflake(Dialect):
"LISTAGG": exp.GroupConcat.from_arg_list,
"NULLIFZERO": _build_if_from_nullifzero,
"OBJECT_CONSTRUCT": _build_object_construct,
"REGEXP_EXTRACT_ALL": _build_regexp_extract(exp.RegexpExtractAll),
"REGEXP_REPLACE": _build_regexp_replace,
"REGEXP_SUBSTR": lambda args: exp.RegexpExtract(
this=seq_get(args, 0),
expression=seq_get(args, 1),
position=seq_get(args, 2),
occurrence=seq_get(args, 3),
parameters=seq_get(args, 4),
group=seq_get(args, 5) or exp.Literal.number(0),
),
"REGEXP_SUBSTR": _build_regexp_extract(exp.RegexpExtract),
"REGEXP_SUBSTR_ALL": _build_regexp_extract(exp.RegexpExtractAll),
"RLIKE": exp.RegexpLike.from_arg_list,
"SQUARE": lambda args: exp.Pow(this=seq_get(args, 0), expression=exp.Literal.number(2)),
"TIMEADD": _build_date_time_add(exp.TimeAdd),
@ -770,6 +823,7 @@ class Snowflake(Dialect):
SUPPORTS_CONVERT_TIMEZONE = True
EXCEPT_INTERSECT_SUPPORT_ALL_CLAUSE = False
SUPPORTS_MEDIAN = True
ARRAY_SIZE_NAME = "ARRAY_SIZE"
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
@ -802,11 +856,13 @@ class Snowflake(Dialect):
),
exp.GroupConcat: rename_func("LISTAGG"),
exp.If: if_sql(name="IFF", false_value="NULL"),
exp.JSONExtractArray: _json_extract_value_array_sql,
exp.JSONExtractScalar: lambda self, e: self.func(
"JSON_EXTRACT_PATH_TEXT", e.this, e.expression
),
exp.JSONObject: lambda self, e: self.func("OBJECT_CONSTRUCT_KEEP_NULL", *e.expressions),
exp.JSONPathRoot: lambda *_: "",
exp.JSONValueArray: _json_extract_value_array_sql,
exp.LogicalAnd: rename_func("BOOLAND_AGG"),
exp.LogicalOr: rename_func("BOOLOR_AGG"),
exp.Map: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"),
@ -823,6 +879,8 @@ class Snowflake(Dialect):
[transforms.add_within_group_for_percentiles]
),
exp.Pivot: transforms.preprocess([_unqualify_unpivot_columns]),
exp.RegexpExtract: _regexpextract_sql,
exp.RegexpExtractAll: _regexpextract_sql,
exp.RegexpILike: _regexpilike_sql,
exp.Rand: rename_func("RANDOM"),
exp.Select: transforms.preprocess(
@ -867,6 +925,9 @@ class Snowflake(Dialect):
exp.VarMap: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"),
exp.WeekOfYear: rename_func("WEEKOFYEAR"),
exp.Xor: rename_func("BOOLXOR"),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost")(
rename_func("EDITDISTANCE")
),
}
SUPPORTED_JSON_PATH_PARTS = {
@ -1009,30 +1070,6 @@ class Snowflake(Dialect):
return f"SHOW {terse}{expression.name}{history}{like}{scope_kind}{scope}{starts_with}{limit}{from_}"
def regexpextract_sql(self, expression: exp.RegexpExtract) -> str:
# Other dialects don't support all of the following parameters, so we need to
# generate default values as necessary to ensure the transpilation is correct
group = expression.args.get("group")
# To avoid generating all these default values, we set group to None if
# it's 0 (also default value) which doesn't trigger the following chain
if group and group.name == "0":
group = None
parameters = expression.args.get("parameters") or (group and exp.Literal.string("c"))
occurrence = expression.args.get("occurrence") or (parameters and exp.Literal.number(1))
position = expression.args.get("position") or (occurrence and exp.Literal.number(1))
return self.func(
"REGEXP_SUBSTR",
expression.this,
expression.expression,
position,
occurrence,
parameters,
group,
)
def describe_sql(self, expression: exp.Describe) -> str:
# Default to table if kind is unknown
kind_value = expression.args.get("kind") or "TABLE"

6
sqlglot/dialects/spark.py
View file

@ -3,7 +3,7 @@ from __future__ import annotations
import typing as t
from sqlglot import exp
from sqlglot.dialects.dialect import rename_func, unit_to_var
from sqlglot.dialects.dialect import rename_func, unit_to_var, timestampdiff_sql, build_date_delta
from sqlglot.dialects.hive import _build_with_ignore_nulls
from sqlglot.dialects.spark2 import Spark2, temporary_storage_provider, _build_as_cast
from sqlglot.helper import ensure_list, seq_get
@ -108,6 +108,7 @@ class Spark(Spark2):
"DATE_ADD": _build_dateadd,
"DATEADD": _build_dateadd,
"TIMESTAMPADD": _build_dateadd,
"TIMESTAMPDIFF": build_date_delta(exp.TimestampDiff),
"DATEDIFF": _build_datediff,
"DATE_DIFF": _build_datediff,
"TIMESTAMP_LTZ": _build_as_cast("TIMESTAMP_LTZ"),
@ -137,6 +138,7 @@ class Spark(Spark2):
PAD_FILL_PATTERN_IS_REQUIRED = False
SUPPORTS_CONVERT_TIMEZONE = True
SUPPORTS_MEDIAN = True
SUPPORTS_UNIX_SECONDS = True
TYPE_MAPPING = {
**Spark2.Generator.TYPE_MAPPING,
@ -166,6 +168,8 @@ class Spark(Spark2):
exp.StartsWith: rename_func("STARTSWITH"),
exp.TsOrDsAdd: _dateadd_sql,
exp.TimestampAdd: _dateadd_sql,
exp.DatetimeDiff: timestampdiff_sql,
exp.TimestampDiff: timestampdiff_sql,
exp.TryCast: lambda self, e: (
self.trycast_sql(e) if e.args.get("safe") else self.cast_sql(e)
),

5
sqlglot/dialects/sqlite.py
View file

@ -18,6 +18,7 @@ from sqlglot.dialects.dialect import (
str_position_sql,
)
from sqlglot.tokens import TokenType
from sqlglot.generator import unsupported_args
def _date_add_sql(self: SQLite.Generator, expression: exp.DateAdd) -> str:
@ -184,7 +185,9 @@ class SQLite(Dialect):
exp.ILike: no_ilike_sql,
exp.JSONExtract: _json_extract_sql,
exp.JSONExtractScalar: arrow_json_extract_sql,
exp.Levenshtein: rename_func("EDITDIST3"),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost", "max_dist")(
rename_func("EDITDIST3")
),
exp.LogicalOr: rename_func("MAX"),
exp.LogicalAnd: rename_func("MIN"),
exp.Pivot: no_pivot_sql,

3
sqlglot/dialects/teradata.py
View file

@ -166,6 +166,7 @@ class Teradata(Dialect):
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"CARDINALITY": exp.ArraySize.from_arg_list,
"RANDOM": lambda args: exp.Rand(lower=seq_get(args, 0), upper=seq_get(args, 1)),
}
@ -227,6 +228,7 @@ class Teradata(Dialect):
LAST_DAY_SUPPORTS_DATE_PART = False
CAN_IMPLEMENT_ARRAY_ANY = True
TZ_TO_WITH_TIME_ZONE = True
ARRAY_SIZE_NAME = "CARDINALITY"
TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING,
@ -246,7 +248,6 @@ class Teradata(Dialect):
**generator.Generator.TRANSFORMS,
exp.ArgMax: rename_func("MAX_BY"),
exp.ArgMin: rename_func("MIN_BY"),
exp.ArraySize: rename_func("CARDINALITY"),
exp.Max: max_or_greatest,
exp.Min: min_or_least,
exp.Pow: lambda self, e: self.binary(e, "**"),

85
sqlglot/expressions.py
View file

@ -767,6 +767,7 @@ class Expression(metaclass=_Expression):
*expressions: t.Optional[ExpOrStr],
dialect: DialectType = None,
copy: bool = True,
wrap: bool = True,
**opts,
) -> Condition:
"""
@ -781,18 +782,22 @@ class Expression(metaclass=_Expression):
If an `Expression` instance is passed, it will be used as-is.
dialect: the dialect used to parse the input expression.
copy: whether to copy the involved expressions (only applies to Expressions).
wrap: whether to wrap the operands in `Paren`s. This is true by default to avoid
precedence issues, but can be turned off when the produced AST is too deep and
causes recursion-related issues.
opts: other options to use to parse the input expressions.
Returns:
The new And condition.
"""
return and_(self, *expressions, dialect=dialect, copy=copy, **opts)
return and_(self, *expressions, dialect=dialect, copy=copy, wrap=wrap, **opts)
def or_(
self,
*expressions: t.Optional[ExpOrStr],
dialect: DialectType = None,
copy: bool = True,
wrap: bool = True,
**opts,
) -> Condition:
"""
@ -807,12 +812,15 @@ class Expression(metaclass=_Expression):
If an `Expression` instance is passed, it will be used as-is.
dialect: the dialect used to parse the input expression.
copy: whether to copy the involved expressions (only applies to Expressions).
wrap: whether to wrap the operands in `Paren`s. This is true by default to avoid
precedence issues, but can be turned off when the produced AST is too deep and
causes recursion-related issues.
opts: other options to use to parse the input expressions.
Returns:
The new Or condition.
"""
return or_(self, *expressions, dialect=dialect, copy=copy, **opts)
return or_(self, *expressions, dialect=dialect, copy=copy, wrap=wrap, **opts)
def not_(self, copy: bool = True):
"""
@ -5939,6 +5947,10 @@ class JSONValue(Expression):
}
class JSONValueArray(Func):
arg_types = {"this": True, "expression": False}
# # https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/JSON_TABLE.html
class JSONTable(Func):
arg_types = {
@ -5995,6 +6007,10 @@ class JSONExtract(Binary, Func):
return self.expression.output_name if not self.expressions else ""
class JSONExtractArray(Func):
arg_types = {"this": True, "expression": False}
class JSONExtractScalar(Binary, Func):
arg_types = {"this": True, "expression": True, "only_json_types": False, "expressions": False}
_sql_names = ["JSON_EXTRACT_SCALAR"]
@ -6233,11 +6249,22 @@ class Reduce(Func):
class RegexpExtract(Func):
arg_types = {
"this": True,
"expression": True, # The pattern
"position": False, # Only start searching the string from this index
"occurrence": False, # Skip the first `occurence-1` matches
"parameters": False, # Flags, eg "i" for case-insensitive
"group": False, # Which group to return
"expression": True,
"position": False,
"occurrence": False,
"parameters": False,
"group": False,
}
class RegexpExtractAll(Func):
arg_types = {
"this": True,
"expression": True,
"position": False,
"occurrence": False,
"parameters": False,
"group": False,
}
@ -6516,6 +6543,10 @@ class UnixToTimeStr(Func):
pass
class UnixSeconds(Func):
pass
class Uuid(Func):
_sql_names = ["UUID", "GEN_RANDOM_UUID", "GENERATE_UUID", "UUID_STRING"]
@ -6898,6 +6929,7 @@ def _combine(
operator: t.Type[Connector],
dialect: DialectType = None,
copy: bool = True,
wrap: bool = True,
**opts,
) -> Expression:
conditions = [
@ -6907,10 +6939,10 @@ def _combine(
]
this, *rest = conditions
if rest:
if rest and wrap:
this = _wrap(this, Connector)
for expression in rest:
this = operator(this=this, expression=_wrap(expression, Connector))
this = operator(this=this, expression=_wrap(expression, Connector) if wrap else expression)
return this
@ -7293,7 +7325,11 @@ def condition(
def and_(
*expressions: t.Optional[ExpOrStr], dialect: DialectType = None, copy: bool = True, **opts
*expressions: t.Optional[ExpOrStr],
dialect: DialectType = None,
copy: bool = True,
wrap: bool = True,
**opts,
) -> Condition:
"""
Combine multiple conditions with an AND logical operator.
@ -7307,16 +7343,23 @@ def and_(
If an Expression instance is passed, this is used as-is.
dialect: the dialect used to parse the input expression.
copy: whether to copy `expressions` (only applies to Expressions).
wrap: whether to wrap the operands in `Paren`s. This is true by default to avoid
precedence issues, but can be turned off when the produced AST is too deep and
causes recursion-related issues.
**opts: other options to use to parse the input expressions.
Returns:
The new condition
"""
return t.cast(Condition, _combine(expressions, And, dialect, copy=copy, **opts))
return t.cast(Condition, _combine(expressions, And, dialect, copy=copy, wrap=wrap, **opts))
def or_(
*expressions: t.Optional[ExpOrStr], dialect: DialectType = None, copy: bool = True, **opts
*expressions: t.Optional[ExpOrStr],
dialect: DialectType = None,
copy: bool = True,
wrap: bool = True,
**opts,
) -> Condition:
"""
Combine multiple conditions with an OR logical operator.
@ -7330,16 +7373,23 @@ def or_(
If an Expression instance is passed, this is used as-is.
dialect: the dialect used to parse the input expression.
copy: whether to copy `expressions` (only applies to Expressions).
wrap: whether to wrap the operands in `Paren`s. This is true by default to avoid
precedence issues, but can be turned off when the produced AST is too deep and
causes recursion-related issues.
**opts: other options to use to parse the input expressions.
Returns:
The new condition
"""
return t.cast(Condition, _combine(expressions, Or, dialect, copy=copy, **opts))
return t.cast(Condition, _combine(expressions, Or, dialect, copy=copy, wrap=wrap, **opts))
def xor(
*expressions: t.Optional[ExpOrStr], dialect: DialectType = None, copy: bool = True, **opts
*expressions: t.Optional[ExpOrStr],
dialect: DialectType = None,
copy: bool = True,
wrap: bool = True,
**opts,
) -> Condition:
"""
Combine multiple conditions with an XOR logical operator.
@ -7353,12 +7403,15 @@ def xor(
If an Expression instance is passed, this is used as-is.
dialect: the dialect used to parse the input expression.
copy: whether to copy `expressions` (only applies to Expressions).
wrap: whether to wrap the operands in `Paren`s. This is true by default to avoid
precedence issues, but can be turned off when the produced AST is too deep and
causes recursion-related issues.
**opts: other options to use to parse the input expressions.
Returns:
The new condition
"""
return t.cast(Condition, _combine(expressions, Xor, dialect, copy=copy, **opts))
return t.cast(Condition, _combine(expressions, Xor, dialect, copy=copy, wrap=wrap, **opts))
def not_(expression: ExpOrStr, dialect: DialectType = None, copy: bool = True, **opts) -> Not:
@ -8052,7 +8105,7 @@ def table_name(table: Table | str, dialect: DialectType = None, identify: bool =
return ".".join(
(
part.sql(dialect=dialect, identify=True, copy=False)
part.sql(dialect=dialect, identify=True, copy=False, comments=False)
if identify or not SAFE_IDENTIFIER_RE.match(part.name)
else part.name
)

39
sqlglot/generator.py
View file

@ -436,9 +436,21 @@ class Generator(metaclass=_Generator):
# Whether MEDIAN(expr) is supported; if not, it will be generated as PERCENTILE_CONT(expr, 0.5)
SUPPORTS_MEDIAN = True
# Whether UNIX_SECONDS(timestamp) is supported
SUPPORTS_UNIX_SECONDS = False
# The name to generate for the JSONPath expression. If `None`, only `this` will be generated
PARSE_JSON_NAME: t.Optional[str] = "PARSE_JSON"
# The function name of the exp.ArraySize expression
ARRAY_SIZE_NAME: str = "ARRAY_LENGTH"
# Whether exp.ArraySize should generate the dimension arg too (valid for Postgres & DuckDB)
# None -> Doesn't support it at all
# False (DuckDB) -> Has backwards-compatible support, but preferably generated without
# True (Postgres) -> Explicitly requires it
ARRAY_SIZE_DIM_REQUIRED: t.Optional[bool] = None
TYPE_MAPPING = {
exp.DataType.Type.NCHAR: "CHAR",
exp.DataType.Type.NVARCHAR: "VARCHAR",
@ -4472,3 +4484,30 @@ class Generator(metaclass=_Generator):
filler = f" {filler}" if filler else ""
with_count = "WITH COUNT" if expression.args.get("with_count") else "WITHOUT COUNT"
return f"TRUNCATE{filler} {with_count}"
def unixseconds_sql(self, expression: exp.UnixSeconds) -> str:
if self.SUPPORTS_UNIX_SECONDS:
return self.function_fallback_sql(expression)
start_ts = exp.cast(
exp.Literal.string("1970-01-01 00:00:00+00"), to=exp.DataType.Type.TIMESTAMPTZ
)
return self.sql(
exp.TimestampDiff(this=expression.this, expression=start_ts, unit=exp.var("SECONDS"))
)
def arraysize_sql(self, expression: exp.ArraySize) -> str:
dim = expression.expression
# For dialects that don't support the dimension arg, we can safely transpile it's default value (1st dimension)
if dim and self.ARRAY_SIZE_DIM_REQUIRED is None:
if not (dim.is_int and dim.name == "1"):
self.unsupported("Cannot transpile dimension argument for ARRAY_LENGTH")
dim = None
# If dimension is required but not specified, default initialize it
if self.ARRAY_SIZE_DIM_REQUIRED and not dim:
dim = exp.Literal.number(1)
return self.func(self.ARRAY_SIZE_NAME, expression.this, dim)

3
sqlglot/parser.py
View file

@ -5138,6 +5138,9 @@ class Parser(metaclass=_Parser):
else:
this = self.expression(exp.Dot, this=this, expression=field)
if field and field.comments:
t.cast(exp.Expression, this).add_comments(field.pop_comments())
this = self._parse_bracket(this)
return self._parse_colon_as_variant_extract(this) if self.COLON_IS_VARIANT_EXTRACT else this

17
sqlglot/transforms.py
View file

@ -348,7 +348,8 @@ def unnest_to_explode(
)
)
for join in expression.args.get("joins") or []:
joins = expression.args.get("joins") or []
for join in list(joins):
join_expr = join.this
is_lateral = isinstance(join_expr, exp.Lateral)
@ -365,9 +366,19 @@ def unnest_to_explode(
has_multi_expr = len(exprs) > 1
exprs = _unnest_zip_exprs(unnest, exprs, has_multi_expr)
expression.args["joins"].remove(join)
joins.remove(join)
alias_cols = alias.columns if alias else []
# # Handle UNNEST to LATERAL VIEW EXPLODE: Exception is raised when there are 0 or > 2 aliases
# Spark LATERAL VIEW EXPLODE requires single alias for array/struct and two for Map type column unlike unnest in trino/presto which can take an arbitrary amount.
# Refs: https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-lateral-view.html
if not has_multi_expr and len(alias_cols) not in (1, 2):
raise UnsupportedError(
"CROSS JOIN UNNEST to LATERAL VIEW EXPLODE transformation requires explicit column aliases"
)
for e, column in zip(exprs, alias_cols):
expression.append(
"laterals",
@ -376,7 +387,7 @@ def unnest_to_explode(
view=True,
alias=exp.TableAlias(
this=alias.this, # type: ignore
columns=alias_cols if unnest_using_arrays_zip else [column], # type: ignore
columns=alias_cols,
),
),
)

131
tests/dialects/test_bigquery.py
View file

@ -10,6 +10,7 @@ from sqlglot import (
exp,
parse,
transpile,
parse_one,
)
from sqlglot.helper import logger as helper_logger
from sqlglot.parser import logger as parser_logger
@ -21,8 +22,6 @@ class TestBigQuery(Validator):
maxDiff = None
def test_bigquery(self):
self.validate_identity("REGEXP_EXTRACT(x, '(?<)')")
self.validate_all(
"EXTRACT(HOUR FROM DATETIME(2008, 12, 25, 15, 30, 00))",
write={
@ -182,7 +181,6 @@ LANGUAGE js AS
self.validate_identity("""CREATE TABLE x (a STRUCT<values ARRAY<INT64>>)""")
self.validate_identity("""CREATE TABLE x (a STRUCT<b STRING OPTIONS (description='b')>)""")
self.validate_identity("CAST(x AS TIMESTAMP)")
self.validate_identity("REGEXP_EXTRACT(`foo`, 'bar: (.+?)', 1, 1)")
self.validate_identity("BEGIN DECLARE y INT64", check_command_warning=True)
self.validate_identity("BEGIN TRANSACTION")
self.validate_identity("COMMIT TRANSACTION")
@ -202,9 +200,24 @@ LANGUAGE js AS
self.validate_identity("CAST(x AS NVARCHAR)", "CAST(x AS STRING)")
self.validate_identity("CAST(x AS TIMESTAMPTZ)", "CAST(x AS TIMESTAMP)")
self.validate_identity("CAST(x AS RECORD)", "CAST(x AS STRUCT)")
self.validate_identity("EDIT_DISTANCE('a', 'a', max_distance => 2)").assert_is(
exp.Levenshtein
self.validate_all(
"EDIT_DISTANCE(col1, col2, max_distance => 3)",
write={
"bigquery": "EDIT_DISTANCE(col1, col2, max_distance => 3)",
"clickhouse": UnsupportedError,
"databricks": UnsupportedError,
"drill": UnsupportedError,
"duckdb": UnsupportedError,
"hive": UnsupportedError,
"postgres": "LEVENSHTEIN_LESS_EQUAL(col1, col2, 3)",
"presto": UnsupportedError,
"snowflake": "EDITDISTANCE(col1, col2, 3)",
"spark": UnsupportedError,
"spark2": UnsupportedError,
"sqlite": UnsupportedError,
},
)
self.validate_identity(
"MERGE INTO dataset.NewArrivals USING (SELECT * FROM UNNEST([('microwave', 10, 'warehouse #1'), ('dryer', 30, 'warehouse #1'), ('oven', 20, 'warehouse #2')])) ON FALSE WHEN NOT MATCHED THEN INSERT ROW WHEN NOT MATCHED BY SOURCE THEN DELETE"
)
@ -314,10 +327,6 @@ LANGUAGE js AS
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) d, COUNT(*) e FOR c IN ('x', 'y'))",
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) AS d, COUNT(*) AS e FOR c IN ('x', 'y'))",
)
self.validate_identity(
r"REGEXP_EXTRACT(svc_plugin_output, r'\\\((.*)')",
r"REGEXP_EXTRACT(svc_plugin_output, '\\\\\\((.*)')",
)
self.validate_identity(
"SELECT CAST(1 AS BYTEINT)",
"SELECT CAST(1 AS INT64)",
@ -1378,14 +1387,6 @@ LANGUAGE js AS
"postgres": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
},
)
self.validate_all(
"SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
write={
"bigquery": "SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
"duckdb": '''SELECT REGEXP_EXTRACT(abc, 'pattern(group)', 1) FROM "table"''',
},
)
self.validate_all(
"SELECT * FROM UNNEST([1]) WITH OFFSET",
write={"bigquery": "SELECT * FROM UNNEST([1]) WITH OFFSET AS offset"},
@ -1602,6 +1603,14 @@ WHERE
"snowflake": """SELECT GET_PATH(PARSE_JSON('{"class": {"students": []}}'), 'class')""",
},
)
self.validate_all(
"""SELECT JSON_VALUE_ARRAY('{"arr": [1, "a"]}', '$.arr')""",
write={
"bigquery": """SELECT JSON_VALUE_ARRAY('{"arr": [1, "a"]}', '$.arr')""",
"duckdb": """SELECT CAST('{"arr": [1, "a"]}' -> '$.arr' AS TEXT[])""",
"snowflake": """SELECT TRANSFORM(GET_PATH(PARSE_JSON('{"arr": [1, "a"]}'), 'arr'), x -> CAST(x AS VARCHAR))""",
},
)
def test_errors(self):
with self.assertRaises(TokenError):
@ -2116,3 +2125,91 @@ OPTIONS (
"snowflake": """SELECT JSON_EXTRACT_PATH_TEXT('{"name": "Jakob", "age": "6"}', 'age')""",
},
)
def test_json_extract_array(self):
for func in ("JSON_QUERY_ARRAY", "JSON_EXTRACT_ARRAY"):
with self.subTest(f"Testing BigQuery's {func}"):
self.validate_all(
f"""SELECT {func}('{{"fruits": [1, "oranges"]}}', '$.fruits')""",
write={
"bigquery": f"""SELECT {func}('{{"fruits": [1, "oranges"]}}', '$.fruits')""",
"duckdb": """SELECT CAST('{"fruits": [1, "oranges"]}' -> '$.fruits' AS JSON[])""",
"snowflake": """SELECT TRANSFORM(GET_PATH(PARSE_JSON('{"fruits": [1, "oranges"]}'), 'fruits'), x -> PARSE_JSON(TO_JSON(x)))""",
},
)
def test_unix_seconds(self):
self.validate_all(
"SELECT UNIX_SECONDS('2008-12-25 15:30:00+00')",
read={
"bigquery": "SELECT UNIX_SECONDS('2008-12-25 15:30:00+00')",
"spark": "SELECT UNIX_SECONDS('2008-12-25 15:30:00+00')",
"databricks": "SELECT UNIX_SECONDS('2008-12-25 15:30:00+00')",
},
write={
"spark": "SELECT UNIX_SECONDS('2008-12-25 15:30:00+00')",
"databricks": "SELECT UNIX_SECONDS('2008-12-25 15:30:00+00')",
"duckdb": "SELECT DATE_DIFF('SECONDS', CAST('1970-01-01 00:00:00+00' AS TIMESTAMPTZ), '2008-12-25 15:30:00+00')",
"snowflake": "SELECT TIMESTAMPDIFF(SECONDS, CAST('1970-01-01 00:00:00+00' AS TIMESTAMPTZ), '2008-12-25 15:30:00+00')",
},
)
for dialect in ("bigquery", "spark", "databricks"):
parse_one("UNIX_SECONDS(col)", dialect=dialect).assert_is(exp.UnixSeconds)
def test_regexp_extract(self):
self.validate_identity("REGEXP_EXTRACT(x, '(?<)')")
self.validate_identity("REGEXP_EXTRACT(`foo`, 'bar: (.+?)', 1, 1)")
self.validate_identity(
r"REGEXP_EXTRACT(svc_plugin_output, r'\\\((.*)')",
r"REGEXP_EXTRACT(svc_plugin_output, '\\\\\\((.*)')",
)
self.validate_identity(
r"REGEXP_SUBSTR(value, pattern, position, occurence)",
r"REGEXP_EXTRACT(value, pattern, position, occurence)",
)
self.validate_all(
"SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
write={
"bigquery": "SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
"duckdb": '''SELECT REGEXP_EXTRACT(abc, 'pattern(group)', 1) FROM "table"''',
},
)
# The pattern does not capture a group (entire regular expression is extracted)
self.validate_all(
"REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
read={
"bigquery": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"trino": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"presto": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"snowflake": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"duckdb": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]', 0)",
"spark": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]', 0)",
"databricks": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]', 0)",
},
write={
"bigquery": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"trino": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"presto": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"snowflake": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]')",
"duckdb": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]', 0)",
"spark": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]', 0)",
"databricks": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', 'a[0-9]', 0)",
},
)
# The pattern does capture >=1 group (the default is to extract the first instance)
self.validate_all(
"REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]')",
write={
"bigquery": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]')",
"trino": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]', 1)",
"presto": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]', 1)",
"snowflake": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]', 1, 1, 'c', 1)",
"duckdb": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]', 1)",
"spark": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]')",
"databricks": "REGEXP_EXTRACT_ALL('a1_a2a3_a4A5a6', '(a)[0-9]')",
},
)

1
tests/dialects/test_clickhouse.py
View file

@ -82,6 +82,7 @@ class TestClickhouse(Validator):
self.validate_identity("SELECT histogram(5)(a)")
self.validate_identity("SELECT groupUniqArray(2)(a)")
self.validate_identity("SELECT exponentialTimeDecayedAvg(60)(a, b)")
self.validate_identity("levenshteinDistance(col1, col2)", "editDistance(col1, col2)")
self.validate_identity("SELECT * FROM foo WHERE x GLOBAL IN (SELECT * FROM bar)")
self.validate_identity("position(haystack, needle)")
self.validate_identity("position(haystack, needle, position)")

110
tests/dialects/test_dialect.py
View file

@ -1444,6 +1444,56 @@ class TestDialect(Validator):
},
)
# UNNEST without column alias
self.validate_all(
"SELECT * FROM x CROSS JOIN UNNEST(y) AS t",
write={
"presto": "SELECT * FROM x CROSS JOIN UNNEST(y) AS t",
"spark": UnsupportedError,
"databricks": UnsupportedError,
},
)
# UNNEST MAP Object into multiple columns, using single alias
self.validate_all(
"SELECT a, b FROM x CROSS JOIN UNNEST(y) AS t (a, b)",
write={
"presto": "SELECT a, b FROM x CROSS JOIN UNNEST(y) AS t(a, b)",
"spark": "SELECT a, b FROM x LATERAL VIEW EXPLODE(y) t AS a, b",
"hive": "SELECT a, b FROM x LATERAL VIEW EXPLODE(y) t AS a, b",
},
)
# Unnest multiple Expression into respective mapped alias
self.validate_all(
"SELECT numbers, animals, n, a FROM (SELECT ARRAY(2, 5) AS numbers, ARRAY('dog', 'cat', 'bird') AS animals UNION ALL SELECT ARRAY(7, 8, 9), ARRAY('cow', 'pig')) AS x CROSS JOIN UNNEST(numbers, animals) AS t(n, a)",
write={
"presto": "SELECT numbers, animals, n, a FROM (SELECT ARRAY[2, 5] AS numbers, ARRAY['dog', 'cat', 'bird'] AS animals UNION ALL SELECT ARRAY[7, 8, 9], ARRAY['cow', 'pig']) AS x CROSS JOIN UNNEST(numbers, animals) AS t(n, a)",
"spark": "SELECT numbers, animals, n, a FROM (SELECT ARRAY(2, 5) AS numbers, ARRAY('dog', 'cat', 'bird') AS animals UNION ALL SELECT ARRAY(7, 8, 9), ARRAY('cow', 'pig')) AS x LATERAL VIEW INLINE(ARRAYS_ZIP(numbers, animals)) t AS n, a",
"hive": UnsupportedError,
},
)
# Unnest column to more then 2 alias (STRUCT)
self.validate_all(
"SELECT a, b, c, d, e FROM x CROSS JOIN UNNEST(y) AS t(a, b, c, d)",
write={
"presto": "SELECT a, b, c, d, e FROM x CROSS JOIN UNNEST(y) AS t(a, b, c, d)",
"spark": UnsupportedError,
"hive": UnsupportedError,
},
)
def test_multiple_chained_unnest(self):
self.validate_all(
"SELECT * FROM x CROSS JOIN UNNEST(a) AS j(lista) CROSS JOIN UNNEST(b) AS k(listb) CROSS JOIN UNNEST(c) AS l(listc)",
write={
"presto": "SELECT * FROM x CROSS JOIN UNNEST(a) AS j(lista) CROSS JOIN UNNEST(b) AS k(listb) CROSS JOIN UNNEST(c) AS l(listc)",
"spark": "SELECT * FROM x LATERAL VIEW EXPLODE(a) j AS lista LATERAL VIEW EXPLODE(b) k AS listb LATERAL VIEW EXPLODE(c) l AS listc",
"hive": "SELECT * FROM x LATERAL VIEW EXPLODE(a) j AS lista LATERAL VIEW EXPLODE(b) k AS listb LATERAL VIEW EXPLODE(c) l AS listc",
},
)
def test_lateral_subquery(self):
self.validate_identity(
"SELECT art FROM tbl1 INNER JOIN LATERAL (SELECT art FROM tbl2) AS tbl2 ON tbl1.art = tbl2.art"
@ -1761,15 +1811,67 @@ class TestDialect(Validator):
)
self.validate_all(
"LEVENSHTEIN(col1, col2)",
write={
read={
"bigquery": "EDIT_DISTANCE(col1, col2)",
"duckdb": "LEVENSHTEIN(col1, col2)",
"clickhouse": "editDistance(col1, col2)",
"drill": "LEVENSHTEIN_DISTANCE(col1, col2)",
"presto": "LEVENSHTEIN_DISTANCE(col1, col2)",
"duckdb": "LEVENSHTEIN(col1, col2)",
"hive": "LEVENSHTEIN(col1, col2)",
"spark": "LEVENSHTEIN(col1, col2)",
"postgres": "LEVENSHTEIN(col1, col2)",
"presto": "LEVENSHTEIN_DISTANCE(col1, col2)",
"snowflake": "EDITDISTANCE(col1, col2)",
"sqlite": "EDITDIST3(col1, col2)",
"trino": "LEVENSHTEIN_DISTANCE(col1, col2)",
},
write={
"bigquery": "EDIT_DISTANCE(col1, col2)",
"clickhouse": "editDistance(col1, col2)",
"drill": "LEVENSHTEIN_DISTANCE(col1, col2)",
"duckdb": "LEVENSHTEIN(col1, col2)",
"hive": "LEVENSHTEIN(col1, col2)",
"spark": "LEVENSHTEIN(col1, col2)",
"postgres": "LEVENSHTEIN(col1, col2)",
"presto": "LEVENSHTEIN_DISTANCE(col1, col2)",
"snowflake": "EDITDISTANCE(col1, col2)",
"sqlite": "EDITDIST3(col1, col2)",
"trino": "LEVENSHTEIN_DISTANCE(col1, col2)",
},
)
self.validate_all(
"LEVENSHTEIN(col1, col2, 1, 2, 3)",
write={
"bigquery": UnsupportedError,
"clickhouse": UnsupportedError,
"drill": UnsupportedError,
"duckdb": UnsupportedError,
"hive": UnsupportedError,
"spark": UnsupportedError,
"postgres": "LEVENSHTEIN(col1, col2, 1, 2, 3)",
"presto": UnsupportedError,
"snowflake": UnsupportedError,
"sqlite": UnsupportedError,
"trino": UnsupportedError,
},
)
self.validate_all(
"LEVENSHTEIN(col1, col2, 1, 2, 3, 4)",
write={
"bigquery": UnsupportedError,
"clickhouse": UnsupportedError,
"drill": UnsupportedError,
"duckdb": UnsupportedError,
"hive": UnsupportedError,
"spark": UnsupportedError,
"postgres": "LEVENSHTEIN_LESS_EQUAL(col1, col2, 1, 2, 3, 4)",
"presto": UnsupportedError,
"snowflake": UnsupportedError,
"sqlite": UnsupportedError,
"trino": UnsupportedError,
},
)
self.validate_all(
"LEVENSHTEIN(coalesce(col1, col2), coalesce(col2, col1))",
write={
@ -3007,7 +3109,7 @@ FROM subquery2""",
"databricks": f"MEDIAN(x){suffix}",
"redshift": f"MEDIAN(x){suffix}",
"oracle": f"MEDIAN(x){suffix}",
"clickhouse": f"MEDIAN(x){suffix}",
"clickhouse": f"median(x){suffix}",
"postgres": f"PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY x){suffix}",
},
)

8
tests/dialects/test_duckdb.py
View file

@ -852,6 +852,7 @@ class TestDuckDB(Validator):
"clickhouse": "DATE_TRUNC('DAY', x)",
},
)
self.validate_identity("EDITDIST3(col1, col2)", "LEVENSHTEIN(col1, col2)")
self.validate_identity("SELECT LENGTH(foo)")
self.validate_identity("SELECT ARRAY[1, 2, 3]", "SELECT [1, 2, 3]")
@ -1153,6 +1154,7 @@ class TestDuckDB(Validator):
self.validate_identity("CAST(x AS BINARY)", "CAST(x AS BLOB)")
self.validate_identity("CAST(x AS VARBINARY)", "CAST(x AS BLOB)")
self.validate_identity("CAST(x AS LOGICAL)", "CAST(x AS BOOLEAN)")
self.validate_identity("""CAST({'i': 1, 's': 'foo'} AS STRUCT("s" TEXT, "i" INT))""")
self.validate_identity(
"CAST(ROW(1, ROW(1)) AS STRUCT(number BIGINT, row STRUCT(number BIGINT)))"
)
@ -1162,11 +1164,11 @@ class TestDuckDB(Validator):
)
self.validate_identity(
"CAST([[STRUCT_PACK(a := 1)]] AS STRUCT(a BIGINT)[][])",
"CAST([[ROW(1)]] AS STRUCT(a BIGINT)[][])",
"CAST([[{'a': 1}]] AS STRUCT(a BIGINT)[][])",
)
self.validate_identity(
"CAST([STRUCT_PACK(a := 1)] AS STRUCT(a BIGINT)[])",
"CAST([ROW(1)] AS STRUCT(a BIGINT)[])",
"CAST([{'a': 1}] AS STRUCT(a BIGINT)[])",
)
self.validate_identity(
"STRUCT_PACK(a := 'b')::json",
@ -1174,7 +1176,7 @@ class TestDuckDB(Validator):
)
self.validate_identity(
"STRUCT_PACK(a := 'b')::STRUCT(a TEXT)",
"CAST(ROW('b') AS STRUCT(a TEXT))",
"CAST({'a': 'b'} AS STRUCT(a TEXT))",
)
self.validate_all(

52
tests/dialects/test_postgres.py
View file

@ -51,7 +51,6 @@ class TestPostgres(Validator):
self.validate_identity("x$")
self.validate_identity("SELECT ARRAY[1, 2, 3]")
self.validate_identity("SELECT ARRAY(SELECT 1)")
self.validate_identity("SELECT ARRAY_LENGTH(ARRAY[1, 2, 3], 1)")
self.validate_identity("STRING_AGG(x, y)")
self.validate_identity("STRING_AGG(x, ',' ORDER BY y)")
self.validate_identity("STRING_AGG(x, ',' ORDER BY y DESC)")
@ -683,6 +682,11 @@ class TestPostgres(Validator):
"""SELECT TRIM(TRAILING ' XXX ' COLLATE "de_DE")""",
"""SELECT RTRIM(' XXX ' COLLATE "de_DE")""",
)
self.validate_identity("LEVENSHTEIN(col1, col2)")
self.validate_identity("LEVENSHTEIN_LESS_EQUAL(col1, col2, 1)")
self.validate_identity("LEVENSHTEIN(col1, col2, 1, 2, 3)")
self.validate_identity("LEVENSHTEIN_LESS_EQUAL(col1, col2, 1, 2, 3, 4)")
self.validate_all(
"""'{"a":1,"b":2}'::json->'b'""",
write={
@ -1237,3 +1241,49 @@ CROSS JOIN JSON_ARRAY_ELEMENTS(CAST(JSON_EXTRACT_PATH(tbox, 'boxes') AS JSON)) A
self.validate_identity(
"""SELECT * FROM table1, ROWS FROM (FUNC1(col1) AS alias1("col1" TEXT)) WITH ORDINALITY AS alias3("col3" INT, "col4" TEXT)"""
)
def test_array_length(self):
self.validate_identity("SELECT ARRAY_LENGTH(ARRAY[1, 2, 3], 1)")
self.validate_all(
"ARRAY_LENGTH(arr, 1)",
read={
"bigquery": "ARRAY_LENGTH(arr)",
"duckdb": "ARRAY_LENGTH(arr)",
"presto": "CARDINALITY(arr)",
"drill": "REPEATED_COUNT(arr)",
"teradata": "CARDINALITY(arr)",
"hive": "SIZE(arr)",
"spark2": "SIZE(arr)",
"spark": "SIZE(arr)",
"databricks": "SIZE(arr)",
},
write={
"duckdb": "ARRAY_LENGTH(arr, 1)",
"presto": "CARDINALITY(arr)",
"teradata": "CARDINALITY(arr)",
"bigquery": "ARRAY_LENGTH(arr)",
"drill": "REPEATED_COUNT(arr)",
"clickhouse": "LENGTH(arr)",
"hive": "SIZE(arr)",
"spark2": "SIZE(arr)",
"spark": "SIZE(arr)",
"databricks": "SIZE(arr)",
},
)
self.validate_all(
"ARRAY_LENGTH(arr, foo)",
write={
"duckdb": "ARRAY_LENGTH(arr, foo)",
"hive": UnsupportedError,
"spark2": UnsupportedError,
"spark": UnsupportedError,
"databricks": UnsupportedError,
"presto": UnsupportedError,
"teradata": UnsupportedError,
"bigquery": UnsupportedError,
"drill": UnsupportedError,
"clickhouse": UnsupportedError,
},
)

16
tests/dialects/test_snowflake.py
View file

@ -946,6 +946,16 @@ WHERE
},
)
self.validate_identity("EDITDISTANCE(col1, col2)")
self.validate_all(
"EDITDISTANCE(col1, col2, 3)",
write={
"bigquery": "EDIT_DISTANCE(col1, col2, max_distance => 3)",
"postgres": "LEVENSHTEIN_LESS_EQUAL(col1, col2, 3)",
"snowflake": "EDITDISTANCE(col1, col2, 3)",
},
)
def test_null_treatment(self):
self.validate_all(
r"SELECT FIRST_VALUE(TABLE1.COLUMN1) OVER (PARTITION BY RANDOM_COLUMN1, RANDOM_COLUMN2 ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS MY_ALIAS FROM TABLE1",
@ -1788,7 +1798,6 @@ FROM persons AS p, LATERAL FLATTEN(input => p.c, path => 'contact') AS _flattene
self.validate_all(
"REGEXP_SUBSTR(subject, pattern, 1, 1, 'c', group)",
read={
"bigquery": "REGEXP_SUBSTR(subject, pattern, 1, 1, 'c', group)",
"duckdb": "REGEXP_EXTRACT(subject, pattern, group)",
"hive": "REGEXP_EXTRACT(subject, pattern, group)",
"presto": "REGEXP_EXTRACT(subject, pattern, group)",
@ -1797,6 +1806,11 @@ FROM persons AS p, LATERAL FLATTEN(input => p.c, path => 'contact') AS _flattene
},
)
self.validate_identity(
"REGEXP_SUBSTR_ALL(subject, pattern)",
"REGEXP_EXTRACT_ALL(subject, pattern)",
)
@mock.patch("sqlglot.generator.logger")
def test_regexp_replace(self, logger):
self.validate_all(

11
tests/dialects/test_spark.py
View file

@ -754,6 +754,17 @@ TBLPROPERTIES (
},
)
self.validate_all(
"SELECT TIMESTAMPDIFF(MONTH, foo, bar)",
read={
"databricks": "SELECT TIMESTAMPDIFF(MONTH, foo, bar)",
},
write={
"spark": "SELECT TIMESTAMPDIFF(MONTH, foo, bar)",
"databricks": "SELECT TIMESTAMPDIFF(MONTH, foo, bar)",
},
)
def test_bool_or(self):
self.validate_all(
"SELECT a, LOGICAL_OR(b) FROM table GROUP BY a",

6
tests/test_expressions.py
View file

@ -216,12 +216,14 @@ class TestExpressions(unittest.TestCase):
self.assertEqual(exp.table_name(exp.to_table("a.b.c.d.e", dialect="bigquery")), "a.b.c.d.e")
self.assertEqual(exp.table_name(exp.to_table("'@foo'", dialect="snowflake")), "'@foo'")
self.assertEqual(exp.table_name(exp.to_table("@foo", dialect="snowflake")), "@foo")
self.assertEqual(exp.table_name(bq_dashed_table, identify=True), '"a-1"."b"."c"')
self.assertEqual(
exp.table_name(parse_one("foo.`{bar,er}`", read="databricks"), dialect="databricks"),
"foo.`{bar,er}`",
)
self.assertEqual(exp.table_name(bq_dashed_table, identify=True), '"a-1"."b"."c"')
self.assertEqual(
exp.table_name(parse_one("/*c*/foo.bar", into=exp.Table), identify=True), '"foo"."bar"'
)
def test_table(self):
self.assertEqual(exp.table_("a", alias="b"), parse_one("select * from a b").find(exp.Table))

36
tests/test_parser.py
View file

@ -340,6 +340,20 @@ class TestParser(unittest.TestCase):
self.assertEqual(expression.expressions[4].comments, [""])
self.assertEqual(expression.expressions[5].comments, [" space"])
expression = parse_one(
"""
SELECT a.column_name --# Comment 1
,b.column_name2, --# Comment 2
b.column_name3 AS NAME3 --# Comment 3
FROM table_name a
JOIN table_name2 b ON a.column_name = b.column_name
"""
)
self.assertEqual(expression.expressions[0].comments, ["# Comment 1"])
self.assertEqual(expression.expressions[1].comments, ["# Comment 2"])
self.assertEqual(expression.expressions[2].comments, ["# Comment 3"])
def test_comments_select_cte(self):
expression = parse_one(
"""
@ -698,20 +712,20 @@ class TestParser(unittest.TestCase):
self.assertEqual(expected_columns, [col.sql(dialect=dialect) for col in columns])
def test_parse_nested(self):
def warn_over_threshold(query: str, max_threshold: float = 0.2):
now = time.time()
query = parse_one("SELECT * FROM a " + ("LEFT JOIN b ON a.id = b.id " * 38))
self.assertIsNotNone(query)
self.assertLessEqual(time.time() - now, 0.1)
ast = parse_one(query)
end = time.time() - now
now = time.time()
query = parse_one("SELECT * FROM a " + ("LEFT JOIN UNNEST(ARRAY[]) " * 15))
self.assertIsNotNone(query)
self.assertLessEqual(time.time() - now, 0.1)
self.assertIsNotNone(ast)
if end >= max_threshold:
parser_logger.warning(
f"Query {query[:100]}... surpassed the time threshold of {max_threshold} seconds"
)
now = time.time()
query = parse_one("SELECT * FROM a " + ("OUTER APPLY (SELECT * FROM b) " * 30))
self.assertIsNotNone(query)
self.assertLessEqual(time.time() - now, 0.1)
warn_over_threshold("SELECT * FROM a " + ("LEFT JOIN b ON a.id = b.id " * 38))
warn_over_threshold("SELECT * FROM a " + ("LEFT JOIN UNNEST(ARRAY[]) " * 15))
warn_over_threshold("SELECT * FROM a " + ("OUTER APPLY (SELECT * FROM b) " * 30))
def test_parse_properties(self):
self.assertEqual(