dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Adding upstream version 21.1.1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-13 21:25:55 +01:00
parent d51e69137a
commit 92ffd7746f
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
91 changed files with 49165 additions and 47854 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
CHANGELOG.md
View file

@ -1,6 +1,30 @@
Changelog
=========
## [v21.1.0] - 2024-02-12
### :sparkles: New Features
- [`e71d489`](https://github.com/tobymao/sqlglot/commit/e71d4899e6744812fdefc2704c66bbd6043b5bc9) - add array and tuple helpers *(commit by [@tobymao](https://github.com/tobymao))*
- [`876e075`](https://github.com/tobymao/sqlglot/commit/876e07580bb2de06b587fc8ad40eb67604ae8507) - **postgres**: root operator closes [#2940](https://github.com/tobymao/sqlglot/pull/2940) *(commit by [@tobymao](https://github.com/tobymao))*
- [`e731276`](https://github.com/tobymao/sqlglot/commit/e731276dd5490a7d294430e0887eebf19e16d28f) - **snowflake**: add support for SHOW USERS *(PR [#2948](https://github.com/tobymao/sqlglot/pull/2948) by [@DanCardin](https://github.com/DanCardin))*
- [`b9d4468`](https://github.com/tobymao/sqlglot/commit/b9d44688c2b785212db635f121b686df02e2dec9) - **tableau**: identifier and quotes closes [#2950](https://github.com/tobymao/sqlglot/pull/2950) *(commit by [@tobymao](https://github.com/tobymao))*
- [`f8d9dbf`](https://github.com/tobymao/sqlglot/commit/f8d9dbf6744f95bf4b7517e8bcc35dd3a6f70c5d) - **sqlite**: add support for IIF *(PR [#2951](https://github.com/tobymao/sqlglot/pull/2951) by [@georgesittas](https://github.com/georgesittas))*
- [`b755551`](https://github.com/tobymao/sqlglot/commit/b7555516c6bf038dc39c4bba2b243839ceb6e3b5) - **clickhouse**: add basic support for system statement *(PR [#2953](https://github.com/tobymao/sqlglot/pull/2953) by [@GaliFFun](https://github.com/GaliFFun))*
### :bug: Bug Fixes
- [`844018b`](https://github.com/tobymao/sqlglot/commit/844018b8d3a3398d746fdc04c966c7e19d311998) - explode_outer to unnest closes [#2941](https://github.com/tobymao/sqlglot/pull/2941) *(commit by [@tobymao](https://github.com/tobymao))*
- [`159da45`](https://github.com/tobymao/sqlglot/commit/159da4523d6eb3ca6853d631bb98dc8f13c7b0fb) - posexplode_outer to unnest *(PR [#2942](https://github.com/tobymao/sqlglot/pull/2942) by [@chelsea-lin](https://github.com/chelsea-lin))*
- [`76d6634`](https://github.com/tobymao/sqlglot/commit/76d66340e566bd9fa8c783f5d311101eb2e80480) - **spark**: CREATE TABLE ... PARTITIONED BY fixes *(PR [#2937](https://github.com/tobymao/sqlglot/pull/2937) by [@barakalon](https://github.com/barakalon))*
- [`d07ddf9`](https://github.com/tobymao/sqlglot/commit/d07ddf9b460c1b6f672fda4f34dc9231419e6c9d) - **optimizer**: remove redundant casts *(PR [#2945](https://github.com/tobymao/sqlglot/pull/2945) by [@barakalon](https://github.com/barakalon))*
- [`b70a394`](https://github.com/tobymao/sqlglot/commit/b70a394222bf209298026fd100f6b9498acf9fff) - if doesn't support different types *(commit by [@tobymao](https://github.com/tobymao))*
- [`6a988e0`](https://github.com/tobymao/sqlglot/commit/6a988e0160022d33623cd036bf84bb0b222c9062) - **bigquery**: fix annotation of timestamp(x) *(PR [#2946](https://github.com/tobymao/sqlglot/pull/2946) by [@georgesittas](https://github.com/georgesittas))*
- [`78e6d0d`](https://github.com/tobymao/sqlglot/commit/78e6d0de83efbff1d3b61c8550db56c1819f7c22) - **optimizer**: qualify_columns optimizations for wide tables *(PR [#2955](https://github.com/tobymao/sqlglot/pull/2955) by [@barakalon](https://github.com/barakalon))*
- [`c20cc70`](https://github.com/tobymao/sqlglot/commit/c20cc70dfc7f6395af157521c7e99074d697beb4) - **redshift**: don't assume Table is an unnested Column if Join has a predicate *(PR [#2956](https://github.com/tobymao/sqlglot/pull/2956) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#2952](https://github.com/tobymao/sqlglot/issues/2952) opened by [@vidit-wisdom](https://github.com/vidit-wisdom)*
### :wrench: Chores
- [`c4524ce`](https://github.com/tobymao/sqlglot/commit/c4524ce1e6a85e16db7ea0289116d0160732dc51) - fix unit test *(commit by [@tobymao](https://github.com/tobymao))*
## [v21.0.2] - 2024-02-08
### :sparkles: New Features
- [`1842c96`](https://github.com/tobymao/sqlglot/commit/1842c96611cadb0227dd3ce8f42457679ab0e08b) - **clickhouse**: add support for LIMIT BY clause *(PR [#2926](https://github.com/tobymao/sqlglot/pull/2926) by [@georgesittas](https://github.com/georgesittas))*
@ -2593,4 +2617,5 @@ Changelog
[v20.11.0]: https://github.com/tobymao/sqlglot/compare/v20.10.0...v20.11.0
[v21.0.0]: https://github.com/tobymao/sqlglot/compare/v20.11.0...v21.0.0
[v21.0.1]: https://github.com/tobymao/sqlglot/compare/v21.0.0...v21.0.1
[v21.0.2]: https://github.com/tobymao/sqlglot/compare/v21.0.1...v21.0.2
[v21.0.2]: https://github.com/tobymao/sqlglot/compare/v21.0.1...v21.0.2
[v21.1.0]: https://github.com/tobymao/sqlglot/compare/v21.0.2...v21.1.0

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

8
docs/sqlglot/_version.html
View file

@ -76,8 +76,8 @@
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;21.0.3.dev0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">21</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">&#39;dev0&#39;</span><span class="p">)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;21.1.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">21</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div>
@ -97,7 +97,7 @@
<section id="version">
<div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;21.0.3.dev0&#39;</span>
<span class="default_value">&#39;21.1.0&#39;</span>
</div>
@ -109,7 +109,7 @@
<section id="version_tuple">
<div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(21, 0, 3, &#39;dev0&#39;)</span>
<span class="default_value">(21, 1, 0)</span>
</div>

482
docs/sqlglot/dataframe/sql.html
View file

@ -607,126 +607,124 @@
</span><span id="SparkSession-71"><a href="#SparkSession-71"><span class="linenos"> 71</span></a> <span class="n">column_mapping</span> <span class="o">=</span> <span class="p">{</span><span class="sa">f</span><span class="s2">&quot;_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">:</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)}</span>
</span><span id="SparkSession-72"><a href="#SparkSession-72"><span class="linenos"> 72</span></a>
</span><span id="SparkSession-73"><a href="#SparkSession-73"><span class="linenos"> 73</span></a> <span class="n">data_expressions</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="SparkSession-74"><a href="#SparkSession-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Tuple</span><span class="p">(</span>
</span><span id="SparkSession-75"><a href="#SparkSession-75"><span class="linenos"> 75</span></a> <span class="n">expressions</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span>
</span><span id="SparkSession-76"><a href="#SparkSession-76"><span class="linenos"> 76</span></a> <span class="nb">map</span><span class="p">(</span>
</span><span id="SparkSession-77"><a href="#SparkSession-77"><span class="linenos"> 77</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
</span><span id="SparkSession-78"><a href="#SparkSession-78"><span class="linenos"> 78</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
</span><span id="SparkSession-79"><a href="#SparkSession-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
</span><span id="SparkSession-80"><a href="#SparkSession-80"><span class="linenos"> 80</span></a> <span class="p">)</span>
</span><span id="SparkSession-81"><a href="#SparkSession-81"><span class="linenos"> 81</span></a> <span class="p">)</span>
</span><span id="SparkSession-82"><a href="#SparkSession-82"><span class="linenos"> 82</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
</span><span id="SparkSession-83"><a href="#SparkSession-83"><span class="linenos"> 83</span></a> <span class="p">]</span>
</span><span id="SparkSession-84"><a href="#SparkSession-84"><span class="linenos"> 84</span></a>
</span><span id="SparkSession-85"><a href="#SparkSession-85"><span class="linenos"> 85</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="SparkSession-86"><a href="#SparkSession-86"><span class="linenos"> 86</span></a> <span class="p">(</span>
</span><span id="SparkSession-87"><a href="#SparkSession-87"><span class="linenos"> 87</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession-88"><a href="#SparkSession-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
</span><span id="SparkSession-89"><a href="#SparkSession-89"><span class="linenos"> 89</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession-90"><a href="#SparkSession-90"><span class="linenos"> 90</span></a> <span class="p">)</span>
</span><span id="SparkSession-91"><a href="#SparkSession-91"><span class="linenos"> 91</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="SparkSession-92"><a href="#SparkSession-92"><span class="linenos"> 92</span></a> <span class="p">]</span>
</span><span id="SparkSession-93"><a href="#SparkSession-93"><span class="linenos"> 93</span></a>
</span><span id="SparkSession-94"><a href="#SparkSession-94"><span class="linenos"> 94</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="SparkSession-95"><a href="#SparkSession-95"><span class="linenos"> 95</span></a> <span class="s2">&quot;expressions&quot;</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
</span><span id="SparkSession-96"><a href="#SparkSession-96"><span class="linenos"> 96</span></a> <span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
</span><span id="SparkSession-97"><a href="#SparkSession-97"><span class="linenos"> 97</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
</span><span id="SparkSession-98"><a href="#SparkSession-98"><span class="linenos"> 98</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
</span><span id="SparkSession-99"><a href="#SparkSession-99"><span class="linenos"> 99</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="SparkSession-100"><a href="#SparkSession-100"><span class="linenos">100</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
</span><span id="SparkSession-101"><a href="#SparkSession-101"><span class="linenos">101</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
</span><span id="SparkSession-102"><a href="#SparkSession-102"><span class="linenos">102</span></a> <span class="p">),</span>
</span><span id="SparkSession-103"><a href="#SparkSession-103"><span class="linenos">103</span></a> <span class="p">),</span>
</span><span id="SparkSession-104"><a href="#SparkSession-104"><span class="linenos">104</span></a> <span class="p">),</span>
</span><span id="SparkSession-105"><a href="#SparkSession-105"><span class="linenos">105</span></a> <span class="p">}</span>
</span><span id="SparkSession-106"><a href="#SparkSession-106"><span class="linenos">106</span></a>
</span><span id="SparkSession-107"><a href="#SparkSession-107"><span class="linenos">107</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
</span><span id="SparkSession-108"><a href="#SparkSession-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
</span><span id="SparkSession-109"><a href="#SparkSession-109"><span class="linenos">109</span></a>
</span><span id="SparkSession-110"><a href="#SparkSession-110"><span class="linenos">110</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">DataFrame</span><span class="p">:</span>
</span><span id="SparkSession-111"><a href="#SparkSession-111"><span class="linenos">111</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession-112"><a href="#SparkSession-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="SparkSession-113"><a href="#SparkSession-113"><span class="linenos">113</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
</span><span id="SparkSession-114"><a href="#SparkSession-114"><span class="linenos">114</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession-115"><a href="#SparkSession-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
</span><span id="SparkSession-116"><a href="#SparkSession-116"><span class="linenos">116</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="SparkSession-117"><a href="#SparkSession-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
</span><span id="SparkSession-118"><a href="#SparkSession-118"><span class="linenos">118</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">))</span>
</span><span id="SparkSession-119"><a href="#SparkSession-119"><span class="linenos">119</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="SparkSession-120"><a href="#SparkSession-120"><span class="linenos">120</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;expression&quot;</span><span class="p">]</span>
</span><span id="SparkSession-121"><a href="#SparkSession-121"><span class="linenos">121</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
</span><span id="SparkSession-122"><a href="#SparkSession-122"><span class="linenos">122</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession-123"><a href="#SparkSession-123"><span class="linenos">123</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="SparkSession-124"><a href="#SparkSession-124"><span class="linenos">124</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="SparkSession-125"><a href="#SparkSession-125"><span class="linenos">125</span></a> <span class="s2">&quot;Unknown expression type provided in the SQL. Please create an issue with the SQL.&quot;</span>
</span><span id="SparkSession-126"><a href="#SparkSession-126"><span class="linenos">126</span></a> <span class="p">)</span>
</span><span id="SparkSession-127"><a href="#SparkSession-127"><span class="linenos">127</span></a> <span class="k">return</span> <span class="n">df</span>
</span><span id="SparkSession-128"><a href="#SparkSession-128"><span class="linenos">128</span></a>
</span><span id="SparkSession-129"><a href="#SparkSession-129"><span class="linenos">129</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-130"><a href="#SparkSession-130"><span class="linenos">130</span></a> <span class="k">def</span> <span class="nf">_auto_incrementing_name</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="SparkSession-131"><a href="#SparkSession-131"><span class="linenos">131</span></a> <span class="n">name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;a</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="SparkSession-132"><a href="#SparkSession-132"><span class="linenos">132</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span> <span class="o">+=</span> <span class="mi">1</span>
</span><span id="SparkSession-133"><a href="#SparkSession-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">name</span>
</span><span id="SparkSession-134"><a href="#SparkSession-134"><span class="linenos">134</span></a>
</span><span id="SparkSession-135"><a href="#SparkSession-135"><span class="linenos">135</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-136"><a href="#SparkSession-136"><span class="linenos">136</span></a> <span class="k">def</span> <span class="nf">_random_branch_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="SparkSession-137"><a href="#SparkSession-137"><span class="linenos">137</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
</span><span id="SparkSession-138"><a href="#SparkSession-138"><span class="linenos">138</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_branch_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
</span><span id="SparkSession-139"><a href="#SparkSession-139"><span class="linenos">139</span></a> <span class="k">return</span> <span class="nb">id</span>
</span><span id="SparkSession-140"><a href="#SparkSession-140"><span class="linenos">140</span></a>
</span><span id="SparkSession-141"><a href="#SparkSession-141"><span class="linenos">141</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-142"><a href="#SparkSession-142"><span class="linenos">142</span></a> <span class="k">def</span> <span class="nf">_random_sequence_id</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="SparkSession-143"><a href="#SparkSession-143"><span class="linenos">143</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
</span><span id="SparkSession-144"><a href="#SparkSession-144"><span class="linenos">144</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_sequence_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
</span><span id="SparkSession-145"><a href="#SparkSession-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="nb">id</span>
</span><span id="SparkSession-146"><a href="#SparkSession-146"><span class="linenos">146</span></a>
</span><span id="SparkSession-147"><a href="#SparkSession-147"><span class="linenos">147</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-148"><a href="#SparkSession-148"><span class="linenos">148</span></a> <span class="k">def</span> <span class="nf">_random_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="SparkSession-149"><a href="#SparkSession-149"><span class="linenos">149</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="s2">&quot;r&quot;</span> <span class="o">+</span> <span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span><span class="o">.</span><span class="n">hex</span>
</span><span id="SparkSession-150"><a href="#SparkSession-150"><span class="linenos">150</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
</span><span id="SparkSession-151"><a href="#SparkSession-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="nb">id</span>
</span><span id="SparkSession-152"><a href="#SparkSession-152"><span class="linenos">152</span></a>
</span><span id="SparkSession-153"><a href="#SparkSession-153"><span class="linenos">153</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-154"><a href="#SparkSession-154"><span class="linenos">154</span></a> <span class="k">def</span> <span class="nf">_join_hint_names</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
</span><span id="SparkSession-155"><a href="#SparkSession-155"><span class="linenos">155</span></a> <span class="k">return</span> <span class="p">{</span><span class="s2">&quot;BROADCAST&quot;</span><span class="p">,</span> <span class="s2">&quot;MERGE&quot;</span><span class="p">,</span> <span class="s2">&quot;SHUFFLE_HASH&quot;</span><span class="p">,</span> <span class="s2">&quot;SHUFFLE_REPLICATE_NL&quot;</span><span class="p">}</span>
</span><span id="SparkSession-156"><a href="#SparkSession-156"><span class="linenos">156</span></a>
</span><span id="SparkSession-157"><a href="#SparkSession-157"><span class="linenos">157</span></a> <span class="k">def</span> <span class="nf">_add_alias_to_mapping</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">sequence_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="SparkSession-158"><a href="#SparkSession-158"><span class="linenos">158</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name_to_sequence_id_mapping</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">sequence_id</span><span class="p">)</span>
</span><span id="SparkSession-159"><a href="#SparkSession-159"><span class="linenos">159</span></a>
</span><span id="SparkSession-160"><a href="#SparkSession-160"><span class="linenos">160</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
</span><span id="SparkSession-161"><a href="#SparkSession-161"><span class="linenos">161</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">&quot;sqlframe.dialect&quot;</span>
</span><span id="SparkSession-162"><a href="#SparkSession-162"><span class="linenos">162</span></a>
</span><span id="SparkSession-163"><a href="#SparkSession-163"><span class="linenos">163</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="SparkSession-164"><a href="#SparkSession-164"><span class="linenos">164</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">&quot;spark&quot;</span>
</span><span id="SparkSession-165"><a href="#SparkSession-165"><span class="linenos">165</span></a>
</span><span id="SparkSession-166"><a href="#SparkSession-166"><span class="linenos">166</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession-167"><a href="#SparkSession-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession-168"><a href="#SparkSession-168"><span class="linenos">168</span></a>
</span><span id="SparkSession-169"><a href="#SparkSession-169"><span class="linenos">169</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
</span><span id="SparkSession-170"><a href="#SparkSession-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession-171"><a href="#SparkSession-171"><span class="linenos">171</span></a>
</span><span id="SparkSession-172"><a href="#SparkSession-172"><span class="linenos">172</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
</span><span id="SparkSession-173"><a href="#SparkSession-173"><span class="linenos">173</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="SparkSession-174"><a href="#SparkSession-174"><span class="linenos">174</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession-175"><a href="#SparkSession-175"><span class="linenos">175</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession-176"><a href="#SparkSession-176"><span class="linenos">176</span></a> <span class="o">*</span><span class="p">,</span>
</span><span id="SparkSession-177"><a href="#SparkSession-177"><span class="linenos">177</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession-178"><a href="#SparkSession-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="SparkSession-179"><a href="#SparkSession-179"><span class="linenos">179</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession-180"><a href="#SparkSession-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
</span><span id="SparkSession-181"><a href="#SparkSession-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
</span><span id="SparkSession-182"><a href="#SparkSession-182"><span class="linenos">182</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
</span><span id="SparkSession-183"><a href="#SparkSession-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
</span><span id="SparkSession-184"><a href="#SparkSession-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession-185"><a href="#SparkSession-185"><span class="linenos">185</span></a>
</span><span id="SparkSession-186"><a href="#SparkSession-186"><span class="linenos">186</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="p">:</span>
</span><span id="SparkSession-187"><a href="#SparkSession-187"><span class="linenos">187</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
</span><span id="SparkSession-188"><a href="#SparkSession-188"><span class="linenos">188</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession-189"><a href="#SparkSession-189"><span class="linenos">189</span></a> <span class="k">return</span> <span class="n">spark</span>
</span><span id="SparkSession-190"><a href="#SparkSession-190"><span class="linenos">190</span></a>
</span><span id="SparkSession-191"><a href="#SparkSession-191"><span class="linenos">191</span></a> <span class="nd">@classproperty</span>
</span><span id="SparkSession-192"><a href="#SparkSession-192"><span class="linenos">192</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession-193"><a href="#SparkSession-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
</span><span id="SparkSession-74"><a href="#SparkSession-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">tuple_</span><span class="p">(</span>
</span><span id="SparkSession-75"><a href="#SparkSession-75"><span class="linenos"> 75</span></a> <span class="o">*</span><span class="nb">map</span><span class="p">(</span>
</span><span id="SparkSession-76"><a href="#SparkSession-76"><span class="linenos"> 76</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
</span><span id="SparkSession-77"><a href="#SparkSession-77"><span class="linenos"> 77</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
</span><span id="SparkSession-78"><a href="#SparkSession-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
</span><span id="SparkSession-79"><a href="#SparkSession-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
</span><span id="SparkSession-80"><a href="#SparkSession-80"><span class="linenos"> 80</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
</span><span id="SparkSession-81"><a href="#SparkSession-81"><span class="linenos"> 81</span></a> <span class="p">]</span>
</span><span id="SparkSession-82"><a href="#SparkSession-82"><span class="linenos"> 82</span></a>
</span><span id="SparkSession-83"><a href="#SparkSession-83"><span class="linenos"> 83</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="SparkSession-84"><a href="#SparkSession-84"><span class="linenos"> 84</span></a> <span class="p">(</span>
</span><span id="SparkSession-85"><a href="#SparkSession-85"><span class="linenos"> 85</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession-86"><a href="#SparkSession-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
</span><span id="SparkSession-87"><a href="#SparkSession-87"><span class="linenos"> 87</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession-88"><a href="#SparkSession-88"><span class="linenos"> 88</span></a> <span class="p">)</span>
</span><span id="SparkSession-89"><a href="#SparkSession-89"><span class="linenos"> 89</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="SparkSession-90"><a href="#SparkSession-90"><span class="linenos"> 90</span></a> <span class="p">]</span>
</span><span id="SparkSession-91"><a href="#SparkSession-91"><span class="linenos"> 91</span></a>
</span><span id="SparkSession-92"><a href="#SparkSession-92"><span class="linenos"> 92</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="SparkSession-93"><a href="#SparkSession-93"><span class="linenos"> 93</span></a> <span class="s2">&quot;expressions&quot;</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
</span><span id="SparkSession-94"><a href="#SparkSession-94"><span class="linenos"> 94</span></a> <span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
</span><span id="SparkSession-95"><a href="#SparkSession-95"><span class="linenos"> 95</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
</span><span id="SparkSession-96"><a href="#SparkSession-96"><span class="linenos"> 96</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
</span><span id="SparkSession-97"><a href="#SparkSession-97"><span class="linenos"> 97</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="SparkSession-98"><a href="#SparkSession-98"><span class="linenos"> 98</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
</span><span id="SparkSession-99"><a href="#SparkSession-99"><span class="linenos"> 99</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
</span><span id="SparkSession-100"><a href="#SparkSession-100"><span class="linenos">100</span></a> <span class="p">),</span>
</span><span id="SparkSession-101"><a href="#SparkSession-101"><span class="linenos">101</span></a> <span class="p">),</span>
</span><span id="SparkSession-102"><a href="#SparkSession-102"><span class="linenos">102</span></a> <span class="p">),</span>
</span><span id="SparkSession-103"><a href="#SparkSession-103"><span class="linenos">103</span></a> <span class="p">}</span>
</span><span id="SparkSession-104"><a href="#SparkSession-104"><span class="linenos">104</span></a>
</span><span id="SparkSession-105"><a href="#SparkSession-105"><span class="linenos">105</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
</span><span id="SparkSession-106"><a href="#SparkSession-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
</span><span id="SparkSession-107"><a href="#SparkSession-107"><span class="linenos">107</span></a>
</span><span id="SparkSession-108"><a href="#SparkSession-108"><span class="linenos">108</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">DataFrame</span><span class="p">:</span>
</span><span id="SparkSession-109"><a href="#SparkSession-109"><span class="linenos">109</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession-110"><a href="#SparkSession-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="SparkSession-111"><a href="#SparkSession-111"><span class="linenos">111</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
</span><span id="SparkSession-112"><a href="#SparkSession-112"><span class="linenos">112</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession-113"><a href="#SparkSession-113"><span class="linenos">113</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
</span><span id="SparkSession-114"><a href="#SparkSession-114"><span class="linenos">114</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="SparkSession-115"><a href="#SparkSession-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
</span><span id="SparkSession-116"><a href="#SparkSession-116"><span class="linenos">116</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">))</span>
</span><span id="SparkSession-117"><a href="#SparkSession-117"><span class="linenos">117</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="SparkSession-118"><a href="#SparkSession-118"><span class="linenos">118</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;expression&quot;</span><span class="p">]</span>
</span><span id="SparkSession-119"><a href="#SparkSession-119"><span class="linenos">119</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
</span><span id="SparkSession-120"><a href="#SparkSession-120"><span class="linenos">120</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession-121"><a href="#SparkSession-121"><span class="linenos">121</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="SparkSession-122"><a href="#SparkSession-122"><span class="linenos">122</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="SparkSession-123"><a href="#SparkSession-123"><span class="linenos">123</span></a> <span class="s2">&quot;Unknown expression type provided in the SQL. Please create an issue with the SQL.&quot;</span>
</span><span id="SparkSession-124"><a href="#SparkSession-124"><span class="linenos">124</span></a> <span class="p">)</span>
</span><span id="SparkSession-125"><a href="#SparkSession-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="n">df</span>
</span><span id="SparkSession-126"><a href="#SparkSession-126"><span class="linenos">126</span></a>
</span><span id="SparkSession-127"><a href="#SparkSession-127"><span class="linenos">127</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-128"><a href="#SparkSession-128"><span class="linenos">128</span></a> <span class="k">def</span> <span class="nf">_auto_incrementing_name</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="SparkSession-129"><a href="#SparkSession-129"><span class="linenos">129</span></a> <span class="n">name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;a</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="SparkSession-130"><a href="#SparkSession-130"><span class="linenos">130</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span> <span class="o">+=</span> <span class="mi">1</span>
</span><span id="SparkSession-131"><a href="#SparkSession-131"><span class="linenos">131</span></a> <span class="k">return</span> <span class="n">name</span>
</span><span id="SparkSession-132"><a href="#SparkSession-132"><span class="linenos">132</span></a>
</span><span id="SparkSession-133"><a href="#SparkSession-133"><span class="linenos">133</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-134"><a href="#SparkSession-134"><span class="linenos">134</span></a> <span class="k">def</span> <span class="nf">_random_branch_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="SparkSession-135"><a href="#SparkSession-135"><span class="linenos">135</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
</span><span id="SparkSession-136"><a href="#SparkSession-136"><span class="linenos">136</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_branch_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
</span><span id="SparkSession-137"><a href="#SparkSession-137"><span class="linenos">137</span></a> <span class="k">return</span> <span class="nb">id</span>
</span><span id="SparkSession-138"><a href="#SparkSession-138"><span class="linenos">138</span></a>
</span><span id="SparkSession-139"><a href="#SparkSession-139"><span class="linenos">139</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-140"><a href="#SparkSession-140"><span class="linenos">140</span></a> <span class="k">def</span> <span class="nf">_random_sequence_id</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="SparkSession-141"><a href="#SparkSession-141"><span class="linenos">141</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
</span><span id="SparkSession-142"><a href="#SparkSession-142"><span class="linenos">142</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_sequence_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
</span><span id="SparkSession-143"><a href="#SparkSession-143"><span class="linenos">143</span></a> <span class="k">return</span> <span class="nb">id</span>
</span><span id="SparkSession-144"><a href="#SparkSession-144"><span class="linenos">144</span></a>
</span><span id="SparkSession-145"><a href="#SparkSession-145"><span class="linenos">145</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-146"><a href="#SparkSession-146"><span class="linenos">146</span></a> <span class="k">def</span> <span class="nf">_random_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="SparkSession-147"><a href="#SparkSession-147"><span class="linenos">147</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="s2">&quot;r&quot;</span> <span class="o">+</span> <span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span><span class="o">.</span><span class="n">hex</span>
</span><span id="SparkSession-148"><a href="#SparkSession-148"><span class="linenos">148</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
</span><span id="SparkSession-149"><a href="#SparkSession-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="nb">id</span>
</span><span id="SparkSession-150"><a href="#SparkSession-150"><span class="linenos">150</span></a>
</span><span id="SparkSession-151"><a href="#SparkSession-151"><span class="linenos">151</span></a> <span class="nd">@property</span>
</span><span id="SparkSession-152"><a href="#SparkSession-152"><span class="linenos">152</span></a> <span class="k">def</span> <span class="nf">_join_hint_names</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
</span><span id="SparkSession-153"><a href="#SparkSession-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="p">{</span><span class="s2">&quot;BROADCAST&quot;</span><span class="p">,</span> <span class="s2">&quot;MERGE&quot;</span><span class="p">,</span> <span class="s2">&quot;SHUFFLE_HASH&quot;</span><span class="p">,</span> <span class="s2">&quot;SHUFFLE_REPLICATE_NL&quot;</span><span class="p">}</span>
</span><span id="SparkSession-154"><a href="#SparkSession-154"><span class="linenos">154</span></a>
</span><span id="SparkSession-155"><a href="#SparkSession-155"><span class="linenos">155</span></a> <span class="k">def</span> <span class="nf">_add_alias_to_mapping</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">sequence_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="SparkSession-156"><a href="#SparkSession-156"><span class="linenos">156</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name_to_sequence_id_mapping</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">sequence_id</span><span class="p">)</span>
</span><span id="SparkSession-157"><a href="#SparkSession-157"><span class="linenos">157</span></a>
</span><span id="SparkSession-158"><a href="#SparkSession-158"><span class="linenos">158</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
</span><span id="SparkSession-159"><a href="#SparkSession-159"><span class="linenos">159</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">&quot;sqlframe.dialect&quot;</span>
</span><span id="SparkSession-160"><a href="#SparkSession-160"><span class="linenos">160</span></a>
</span><span id="SparkSession-161"><a href="#SparkSession-161"><span class="linenos">161</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="SparkSession-162"><a href="#SparkSession-162"><span class="linenos">162</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">&quot;spark&quot;</span>
</span><span id="SparkSession-163"><a href="#SparkSession-163"><span class="linenos">163</span></a>
</span><span id="SparkSession-164"><a href="#SparkSession-164"><span class="linenos">164</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession-165"><a href="#SparkSession-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession-166"><a href="#SparkSession-166"><span class="linenos">166</span></a>
</span><span id="SparkSession-167"><a href="#SparkSession-167"><span class="linenos">167</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
</span><span id="SparkSession-168"><a href="#SparkSession-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession-169"><a href="#SparkSession-169"><span class="linenos">169</span></a>
</span><span id="SparkSession-170"><a href="#SparkSession-170"><span class="linenos">170</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
</span><span id="SparkSession-171"><a href="#SparkSession-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="SparkSession-172"><a href="#SparkSession-172"><span class="linenos">172</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession-173"><a href="#SparkSession-173"><span class="linenos">173</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession-174"><a href="#SparkSession-174"><span class="linenos">174</span></a> <span class="o">*</span><span class="p">,</span>
</span><span id="SparkSession-175"><a href="#SparkSession-175"><span class="linenos">175</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession-176"><a href="#SparkSession-176"><span class="linenos">176</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="SparkSession-177"><a href="#SparkSession-177"><span class="linenos">177</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession-178"><a href="#SparkSession-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
</span><span id="SparkSession-179"><a href="#SparkSession-179"><span class="linenos">179</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
</span><span id="SparkSession-180"><a href="#SparkSession-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
</span><span id="SparkSession-181"><a href="#SparkSession-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
</span><span id="SparkSession-182"><a href="#SparkSession-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession-183"><a href="#SparkSession-183"><span class="linenos">183</span></a>
</span><span id="SparkSession-184"><a href="#SparkSession-184"><span class="linenos">184</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="p">:</span>
</span><span id="SparkSession-185"><a href="#SparkSession-185"><span class="linenos">185</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
</span><span id="SparkSession-186"><a href="#SparkSession-186"><span class="linenos">186</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession-187"><a href="#SparkSession-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">spark</span>
</span><span id="SparkSession-188"><a href="#SparkSession-188"><span class="linenos">188</span></a>
</span><span id="SparkSession-189"><a href="#SparkSession-189"><span class="linenos">189</span></a> <span class="nd">@classproperty</span>
</span><span id="SparkSession-190"><a href="#SparkSession-190"><span class="linenos">190</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession-191"><a href="#SparkSession-191"><span class="linenos">191</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
</span></pre></div>
@ -786,7 +784,7 @@
<div class="attr function">
<span class="def">def</span>
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763522736&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763522736&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595765140864&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981980193248&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981980193248&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978717392&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
@ -820,41 +818,39 @@
</span><span id="SparkSession.createDataFrame-71"><a href="#SparkSession.createDataFrame-71"><span class="linenos"> 71</span></a> <span class="n">column_mapping</span> <span class="o">=</span> <span class="p">{</span><span class="sa">f</span><span class="s2">&quot;_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">:</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)}</span>
</span><span id="SparkSession.createDataFrame-72"><a href="#SparkSession.createDataFrame-72"><span class="linenos"> 72</span></a>
</span><span id="SparkSession.createDataFrame-73"><a href="#SparkSession.createDataFrame-73"><span class="linenos"> 73</span></a> <span class="n">data_expressions</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="SparkSession.createDataFrame-74"><a href="#SparkSession.createDataFrame-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Tuple</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-75"><a href="#SparkSession.createDataFrame-75"><span class="linenos"> 75</span></a> <span class="n">expressions</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-76"><a href="#SparkSession.createDataFrame-76"><span class="linenos"> 76</span></a> <span class="nb">map</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-77"><a href="#SparkSession.createDataFrame-77"><span class="linenos"> 77</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
</span><span id="SparkSession.createDataFrame-78"><a href="#SparkSession.createDataFrame-78"><span class="linenos"> 78</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
</span><span id="SparkSession.createDataFrame-79"><a href="#SparkSession.createDataFrame-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-80"><a href="#SparkSession.createDataFrame-80"><span class="linenos"> 80</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-81"><a href="#SparkSession.createDataFrame-81"><span class="linenos"> 81</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-82"><a href="#SparkSession.createDataFrame-82"><span class="linenos"> 82</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
</span><span id="SparkSession.createDataFrame-83"><a href="#SparkSession.createDataFrame-83"><span class="linenos"> 83</span></a> <span class="p">]</span>
</span><span id="SparkSession.createDataFrame-84"><a href="#SparkSession.createDataFrame-84"><span class="linenos"> 84</span></a>
</span><span id="SparkSession.createDataFrame-85"><a href="#SparkSession.createDataFrame-85"><span class="linenos"> 85</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="SparkSession.createDataFrame-86"><a href="#SparkSession.createDataFrame-86"><span class="linenos"> 86</span></a> <span class="p">(</span>
</span><span id="SparkSession.createDataFrame-87"><a href="#SparkSession.createDataFrame-87"><span class="linenos"> 87</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession.createDataFrame-88"><a href="#SparkSession.createDataFrame-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
</span><span id="SparkSession.createDataFrame-89"><a href="#SparkSession.createDataFrame-89"><span class="linenos"> 89</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession.createDataFrame-90"><a href="#SparkSession.createDataFrame-90"><span class="linenos"> 90</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-91"><a href="#SparkSession.createDataFrame-91"><span class="linenos"> 91</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="SparkSession.createDataFrame-92"><a href="#SparkSession.createDataFrame-92"><span class="linenos"> 92</span></a> <span class="p">]</span>
</span><span id="SparkSession.createDataFrame-93"><a href="#SparkSession.createDataFrame-93"><span class="linenos"> 93</span></a>
</span><span id="SparkSession.createDataFrame-94"><a href="#SparkSession.createDataFrame-94"><span class="linenos"> 94</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="SparkSession.createDataFrame-95"><a href="#SparkSession.createDataFrame-95"><span class="linenos"> 95</span></a> <span class="s2">&quot;expressions&quot;</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
</span><span id="SparkSession.createDataFrame-96"><a href="#SparkSession.createDataFrame-96"><span class="linenos"> 96</span></a> <span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-97"><a href="#SparkSession.createDataFrame-97"><span class="linenos"> 97</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-98"><a href="#SparkSession.createDataFrame-98"><span class="linenos"> 98</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
</span><span id="SparkSession.createDataFrame-99"><a href="#SparkSession.createDataFrame-99"><span class="linenos"> 99</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-100"><a href="#SparkSession.createDataFrame-100"><span class="linenos">100</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
</span><span id="SparkSession.createDataFrame-101"><a href="#SparkSession.createDataFrame-101"><span class="linenos">101</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
</span><span id="SparkSession.createDataFrame-102"><a href="#SparkSession.createDataFrame-102"><span class="linenos">102</span></a> <span class="p">),</span>
</span><span id="SparkSession.createDataFrame-103"><a href="#SparkSession.createDataFrame-103"><span class="linenos">103</span></a> <span class="p">),</span>
</span><span id="SparkSession.createDataFrame-104"><a href="#SparkSession.createDataFrame-104"><span class="linenos">104</span></a> <span class="p">),</span>
</span><span id="SparkSession.createDataFrame-105"><a href="#SparkSession.createDataFrame-105"><span class="linenos">105</span></a> <span class="p">}</span>
</span><span id="SparkSession.createDataFrame-106"><a href="#SparkSession.createDataFrame-106"><span class="linenos">106</span></a>
</span><span id="SparkSession.createDataFrame-107"><a href="#SparkSession.createDataFrame-107"><span class="linenos">107</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
</span><span id="SparkSession.createDataFrame-108"><a href="#SparkSession.createDataFrame-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
</span><span id="SparkSession.createDataFrame-74"><a href="#SparkSession.createDataFrame-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">tuple_</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-75"><a href="#SparkSession.createDataFrame-75"><span class="linenos"> 75</span></a> <span class="o">*</span><span class="nb">map</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-76"><a href="#SparkSession.createDataFrame-76"><span class="linenos"> 76</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
</span><span id="SparkSession.createDataFrame-77"><a href="#SparkSession.createDataFrame-77"><span class="linenos"> 77</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
</span><span id="SparkSession.createDataFrame-78"><a href="#SparkSession.createDataFrame-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-79"><a href="#SparkSession.createDataFrame-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-80"><a href="#SparkSession.createDataFrame-80"><span class="linenos"> 80</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
</span><span id="SparkSession.createDataFrame-81"><a href="#SparkSession.createDataFrame-81"><span class="linenos"> 81</span></a> <span class="p">]</span>
</span><span id="SparkSession.createDataFrame-82"><a href="#SparkSession.createDataFrame-82"><span class="linenos"> 82</span></a>
</span><span id="SparkSession.createDataFrame-83"><a href="#SparkSession.createDataFrame-83"><span class="linenos"> 83</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="SparkSession.createDataFrame-84"><a href="#SparkSession.createDataFrame-84"><span class="linenos"> 84</span></a> <span class="p">(</span>
</span><span id="SparkSession.createDataFrame-85"><a href="#SparkSession.createDataFrame-85"><span class="linenos"> 85</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession.createDataFrame-86"><a href="#SparkSession.createDataFrame-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
</span><span id="SparkSession.createDataFrame-87"><a href="#SparkSession.createDataFrame-87"><span class="linenos"> 87</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
</span><span id="SparkSession.createDataFrame-88"><a href="#SparkSession.createDataFrame-88"><span class="linenos"> 88</span></a> <span class="p">)</span>
</span><span id="SparkSession.createDataFrame-89"><a href="#SparkSession.createDataFrame-89"><span class="linenos"> 89</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="SparkSession.createDataFrame-90"><a href="#SparkSession.createDataFrame-90"><span class="linenos"> 90</span></a> <span class="p">]</span>
</span><span id="SparkSession.createDataFrame-91"><a href="#SparkSession.createDataFrame-91"><span class="linenos"> 91</span></a>
</span><span id="SparkSession.createDataFrame-92"><a href="#SparkSession.createDataFrame-92"><span class="linenos"> 92</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="SparkSession.createDataFrame-93"><a href="#SparkSession.createDataFrame-93"><span class="linenos"> 93</span></a> <span class="s2">&quot;expressions&quot;</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
</span><span id="SparkSession.createDataFrame-94"><a href="#SparkSession.createDataFrame-94"><span class="linenos"> 94</span></a> <span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-95"><a href="#SparkSession.createDataFrame-95"><span class="linenos"> 95</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-96"><a href="#SparkSession.createDataFrame-96"><span class="linenos"> 96</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
</span><span id="SparkSession.createDataFrame-97"><a href="#SparkSession.createDataFrame-97"><span class="linenos"> 97</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="SparkSession.createDataFrame-98"><a href="#SparkSession.createDataFrame-98"><span class="linenos"> 98</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
</span><span id="SparkSession.createDataFrame-99"><a href="#SparkSession.createDataFrame-99"><span class="linenos"> 99</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
</span><span id="SparkSession.createDataFrame-100"><a href="#SparkSession.createDataFrame-100"><span class="linenos">100</span></a> <span class="p">),</span>
</span><span id="SparkSession.createDataFrame-101"><a href="#SparkSession.createDataFrame-101"><span class="linenos">101</span></a> <span class="p">),</span>
</span><span id="SparkSession.createDataFrame-102"><a href="#SparkSession.createDataFrame-102"><span class="linenos">102</span></a> <span class="p">),</span>
</span><span id="SparkSession.createDataFrame-103"><a href="#SparkSession.createDataFrame-103"><span class="linenos">103</span></a> <span class="p">}</span>
</span><span id="SparkSession.createDataFrame-104"><a href="#SparkSession.createDataFrame-104"><span class="linenos">104</span></a>
</span><span id="SparkSession.createDataFrame-105"><a href="#SparkSession.createDataFrame-105"><span class="linenos">105</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
</span><span id="SparkSession.createDataFrame-106"><a href="#SparkSession.createDataFrame-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
</span></pre></div>
@ -872,24 +868,24 @@
</div>
<a class="headerlink" href="#SparkSession.sql"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.sql-110"><a href="#SparkSession.sql-110"><span class="linenos">110</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">DataFrame</span><span class="p">:</span>
</span><span id="SparkSession.sql-111"><a href="#SparkSession.sql-111"><span class="linenos">111</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession.sql-112"><a href="#SparkSession.sql-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="SparkSession.sql-113"><a href="#SparkSession.sql-113"><span class="linenos">113</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
</span><span id="SparkSession.sql-114"><a href="#SparkSession.sql-114"><span class="linenos">114</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession.sql-115"><a href="#SparkSession.sql-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
</span><span id="SparkSession.sql-116"><a href="#SparkSession.sql-116"><span class="linenos">116</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="SparkSession.sql-117"><a href="#SparkSession.sql-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
</span><span id="SparkSession.sql-118"><a href="#SparkSession.sql-118"><span class="linenos">118</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">))</span>
</span><span id="SparkSession.sql-119"><a href="#SparkSession.sql-119"><span class="linenos">119</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="SparkSession.sql-120"><a href="#SparkSession.sql-120"><span class="linenos">120</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;expression&quot;</span><span class="p">]</span>
</span><span id="SparkSession.sql-121"><a href="#SparkSession.sql-121"><span class="linenos">121</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
</span><span id="SparkSession.sql-122"><a href="#SparkSession.sql-122"><span class="linenos">122</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession.sql-123"><a href="#SparkSession.sql-123"><span class="linenos">123</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="SparkSession.sql-124"><a href="#SparkSession.sql-124"><span class="linenos">124</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="SparkSession.sql-125"><a href="#SparkSession.sql-125"><span class="linenos">125</span></a> <span class="s2">&quot;Unknown expression type provided in the SQL. Please create an issue with the SQL.&quot;</span>
</span><span id="SparkSession.sql-126"><a href="#SparkSession.sql-126"><span class="linenos">126</span></a> <span class="p">)</span>
</span><span id="SparkSession.sql-127"><a href="#SparkSession.sql-127"><span class="linenos">127</span></a> <span class="k">return</span> <span class="n">df</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.sql-108"><a href="#SparkSession.sql-108"><span class="linenos">108</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">DataFrame</span><span class="p">:</span>
</span><span id="SparkSession.sql-109"><a href="#SparkSession.sql-109"><span class="linenos">109</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession.sql-110"><a href="#SparkSession.sql-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="SparkSession.sql-111"><a href="#SparkSession.sql-111"><span class="linenos">111</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
</span><span id="SparkSession.sql-112"><a href="#SparkSession.sql-112"><span class="linenos">112</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession.sql-113"><a href="#SparkSession.sql-113"><span class="linenos">113</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
</span><span id="SparkSession.sql-114"><a href="#SparkSession.sql-114"><span class="linenos">114</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="SparkSession.sql-115"><a href="#SparkSession.sql-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
</span><span id="SparkSession.sql-116"><a href="#SparkSession.sql-116"><span class="linenos">116</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">))</span>
</span><span id="SparkSession.sql-117"><a href="#SparkSession.sql-117"><span class="linenos">117</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="SparkSession.sql-118"><a href="#SparkSession.sql-118"><span class="linenos">118</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;expression&quot;</span><span class="p">]</span>
</span><span id="SparkSession.sql-119"><a href="#SparkSession.sql-119"><span class="linenos">119</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
</span><span id="SparkSession.sql-120"><a href="#SparkSession.sql-120"><span class="linenos">120</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
</span><span id="SparkSession.sql-121"><a href="#SparkSession.sql-121"><span class="linenos">121</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="SparkSession.sql-122"><a href="#SparkSession.sql-122"><span class="linenos">122</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
</span><span id="SparkSession.sql-123"><a href="#SparkSession.sql-123"><span class="linenos">123</span></a> <span class="s2">&quot;Unknown expression type provided in the SQL. Please create an issue with the SQL.&quot;</span>
</span><span id="SparkSession.sql-124"><a href="#SparkSession.sql-124"><span class="linenos">124</span></a> <span class="p">)</span>
</span><span id="SparkSession.sql-125"><a href="#SparkSession.sql-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="n">df</span>
</span></pre></div>
@ -905,9 +901,9 @@
</div>
<a class="headerlink" href="#SparkSession.builder"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.builder-191"><a href="#SparkSession.builder-191"><span class="linenos">191</span></a> <span class="nd">@classproperty</span>
</span><span id="SparkSession.builder-192"><a href="#SparkSession.builder-192"><span class="linenos">192</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.builder-193"><a href="#SparkSession.builder-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.builder-189"><a href="#SparkSession.builder-189"><span class="linenos">189</span></a> <span class="nd">@classproperty</span>
</span><span id="SparkSession.builder-190"><a href="#SparkSession.builder-190"><span class="linenos">190</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.builder-191"><a href="#SparkSession.builder-191"><span class="linenos">191</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
</span></pre></div>
@ -926,36 +922,36 @@
</div>
<a class="headerlink" href="#SparkSession.Builder"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder-160"><a href="#SparkSession.Builder-160"><span class="linenos">160</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder-161"><a href="#SparkSession.Builder-161"><span class="linenos">161</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">&quot;sqlframe.dialect&quot;</span>
</span><span id="SparkSession.Builder-162"><a href="#SparkSession.Builder-162"><span class="linenos">162</span></a>
</span><span id="SparkSession.Builder-163"><a href="#SparkSession.Builder-163"><span class="linenos">163</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="SparkSession.Builder-164"><a href="#SparkSession.Builder-164"><span class="linenos">164</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">&quot;spark&quot;</span>
</span><span id="SparkSession.Builder-165"><a href="#SparkSession.Builder-165"><span class="linenos">165</span></a>
</span><span id="SparkSession.Builder-166"><a href="#SparkSession.Builder-166"><span class="linenos">166</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder-167"><a href="#SparkSession.Builder-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession.Builder-168"><a href="#SparkSession.Builder-168"><span class="linenos">168</span></a>
</span><span id="SparkSession.Builder-169"><a href="#SparkSession.Builder-169"><span class="linenos">169</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
</span><span id="SparkSession.Builder-170"><a href="#SparkSession.Builder-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession.Builder-171"><a href="#SparkSession.Builder-171"><span class="linenos">171</span></a>
</span><span id="SparkSession.Builder-172"><a href="#SparkSession.Builder-172"><span class="linenos">172</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
</span><span id="SparkSession.Builder-173"><a href="#SparkSession.Builder-173"><span class="linenos">173</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="SparkSession.Builder-174"><a href="#SparkSession.Builder-174"><span class="linenos">174</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder-175"><a href="#SparkSession.Builder-175"><span class="linenos">175</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder-176"><a href="#SparkSession.Builder-176"><span class="linenos">176</span></a> <span class="o">*</span><span class="p">,</span>
</span><span id="SparkSession.Builder-177"><a href="#SparkSession.Builder-177"><span class="linenos">177</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder-178"><a href="#SparkSession.Builder-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="SparkSession.Builder-179"><a href="#SparkSession.Builder-179"><span class="linenos">179</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder-180"><a href="#SparkSession.Builder-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
</span><span id="SparkSession.Builder-181"><a href="#SparkSession.Builder-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
</span><span id="SparkSession.Builder-182"><a href="#SparkSession.Builder-182"><span class="linenos">182</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
</span><span id="SparkSession.Builder-183"><a href="#SparkSession.Builder-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
</span><span id="SparkSession.Builder-184"><a href="#SparkSession.Builder-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession.Builder-185"><a href="#SparkSession.Builder-185"><span class="linenos">185</span></a>
</span><span id="SparkSession.Builder-186"><a href="#SparkSession.Builder-186"><span class="linenos">186</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="p">:</span>
</span><span id="SparkSession.Builder-187"><a href="#SparkSession.Builder-187"><span class="linenos">187</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
</span><span id="SparkSession.Builder-188"><a href="#SparkSession.Builder-188"><span class="linenos">188</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession.Builder-189"><a href="#SparkSession.Builder-189"><span class="linenos">189</span></a> <span class="k">return</span> <span class="n">spark</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder-158"><a href="#SparkSession.Builder-158"><span class="linenos">158</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder-159"><a href="#SparkSession.Builder-159"><span class="linenos">159</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">&quot;sqlframe.dialect&quot;</span>
</span><span id="SparkSession.Builder-160"><a href="#SparkSession.Builder-160"><span class="linenos">160</span></a>
</span><span id="SparkSession.Builder-161"><a href="#SparkSession.Builder-161"><span class="linenos">161</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="SparkSession.Builder-162"><a href="#SparkSession.Builder-162"><span class="linenos">162</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">&quot;spark&quot;</span>
</span><span id="SparkSession.Builder-163"><a href="#SparkSession.Builder-163"><span class="linenos">163</span></a>
</span><span id="SparkSession.Builder-164"><a href="#SparkSession.Builder-164"><span class="linenos">164</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder-165"><a href="#SparkSession.Builder-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession.Builder-166"><a href="#SparkSession.Builder-166"><span class="linenos">166</span></a>
</span><span id="SparkSession.Builder-167"><a href="#SparkSession.Builder-167"><span class="linenos">167</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
</span><span id="SparkSession.Builder-168"><a href="#SparkSession.Builder-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession.Builder-169"><a href="#SparkSession.Builder-169"><span class="linenos">169</span></a>
</span><span id="SparkSession.Builder-170"><a href="#SparkSession.Builder-170"><span class="linenos">170</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
</span><span id="SparkSession.Builder-171"><a href="#SparkSession.Builder-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="SparkSession.Builder-172"><a href="#SparkSession.Builder-172"><span class="linenos">172</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder-173"><a href="#SparkSession.Builder-173"><span class="linenos">173</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder-174"><a href="#SparkSession.Builder-174"><span class="linenos">174</span></a> <span class="o">*</span><span class="p">,</span>
</span><span id="SparkSession.Builder-175"><a href="#SparkSession.Builder-175"><span class="linenos">175</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder-176"><a href="#SparkSession.Builder-176"><span class="linenos">176</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="SparkSession.Builder-177"><a href="#SparkSession.Builder-177"><span class="linenos">177</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder-178"><a href="#SparkSession.Builder-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
</span><span id="SparkSession.Builder-179"><a href="#SparkSession.Builder-179"><span class="linenos">179</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
</span><span id="SparkSession.Builder-180"><a href="#SparkSession.Builder-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
</span><span id="SparkSession.Builder-181"><a href="#SparkSession.Builder-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
</span><span id="SparkSession.Builder-182"><a href="#SparkSession.Builder-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="SparkSession.Builder-183"><a href="#SparkSession.Builder-183"><span class="linenos">183</span></a>
</span><span id="SparkSession.Builder-184"><a href="#SparkSession.Builder-184"><span class="linenos">184</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="p">:</span>
</span><span id="SparkSession.Builder-185"><a href="#SparkSession.Builder-185"><span class="linenos">185</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
</span><span id="SparkSession.Builder-186"><a href="#SparkSession.Builder-186"><span class="linenos">186</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession.Builder-187"><a href="#SparkSession.Builder-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">spark</span>
</span></pre></div>
@ -995,19 +991,19 @@
</div>
<a class="headerlink" href="#SparkSession.Builder.config"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.config-172"><a href="#SparkSession.Builder.config-172"><span class="linenos">172</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
</span><span id="SparkSession.Builder.config-173"><a href="#SparkSession.Builder.config-173"><span class="linenos">173</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-174"><a href="#SparkSession.Builder.config-174"><span class="linenos">174</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-175"><a href="#SparkSession.Builder.config-175"><span class="linenos">175</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-176"><a href="#SparkSession.Builder.config-176"><span class="linenos">176</span></a> <span class="o">*</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-177"><a href="#SparkSession.Builder.config-177"><span class="linenos">177</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-178"><a href="#SparkSession.Builder.config-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-179"><a href="#SparkSession.Builder.config-179"><span class="linenos">179</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder.config-180"><a href="#SparkSession.Builder.config-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
</span><span id="SparkSession.Builder.config-181"><a href="#SparkSession.Builder.config-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
</span><span id="SparkSession.Builder.config-182"><a href="#SparkSession.Builder.config-182"><span class="linenos">182</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
</span><span id="SparkSession.Builder.config-183"><a href="#SparkSession.Builder.config-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
</span><span id="SparkSession.Builder.config-184"><a href="#SparkSession.Builder.config-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="bp">self</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.config-170"><a href="#SparkSession.Builder.config-170"><span class="linenos">170</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
</span><span id="SparkSession.Builder.config-171"><a href="#SparkSession.Builder.config-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-172"><a href="#SparkSession.Builder.config-172"><span class="linenos">172</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-173"><a href="#SparkSession.Builder.config-173"><span class="linenos">173</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-174"><a href="#SparkSession.Builder.config-174"><span class="linenos">174</span></a> <span class="o">*</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-175"><a href="#SparkSession.Builder.config-175"><span class="linenos">175</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-176"><a href="#SparkSession.Builder.config-176"><span class="linenos">176</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="SparkSession.Builder.config-177"><a href="#SparkSession.Builder.config-177"><span class="linenos">177</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
</span><span id="SparkSession.Builder.config-178"><a href="#SparkSession.Builder.config-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
</span><span id="SparkSession.Builder.config-179"><a href="#SparkSession.Builder.config-179"><span class="linenos">179</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
</span><span id="SparkSession.Builder.config-180"><a href="#SparkSession.Builder.config-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
</span><span id="SparkSession.Builder.config-181"><a href="#SparkSession.Builder.config-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
</span><span id="SparkSession.Builder.config-182"><a href="#SparkSession.Builder.config-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="bp">self</span>
</span></pre></div>
@ -1025,10 +1021,10 @@
</div>
<a class="headerlink" href="#SparkSession.Builder.getOrCreate"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.getOrCreate-186"><a href="#SparkSession.Builder.getOrCreate-186"><span class="linenos">186</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="p">:</span>
</span><span id="SparkSession.Builder.getOrCreate-187"><a href="#SparkSession.Builder.getOrCreate-187"><span class="linenos">187</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
</span><span id="SparkSession.Builder.getOrCreate-188"><a href="#SparkSession.Builder.getOrCreate-188"><span class="linenos">188</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession.Builder.getOrCreate-189"><a href="#SparkSession.Builder.getOrCreate-189"><span class="linenos">189</span></a> <span class="k">return</span> <span class="n">spark</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.getOrCreate-184"><a href="#SparkSession.Builder.getOrCreate-184"><span class="linenos">184</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">SparkSession</span><span class="p">:</span>
</span><span id="SparkSession.Builder.getOrCreate-185"><a href="#SparkSession.Builder.getOrCreate-185"><span class="linenos">185</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
</span><span id="SparkSession.Builder.getOrCreate-186"><a href="#SparkSession.Builder.getOrCreate-186"><span class="linenos">186</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="SparkSession.Builder.getOrCreate-187"><a href="#SparkSession.Builder.getOrCreate-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">spark</span>
</span></pre></div>
@ -1844,7 +1840,7 @@
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function">
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595768209808&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o">&lt;</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595768526640&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981983176912&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o">&lt;</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981983444528&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
@ -2092,7 +2088,7 @@
<div class="attr function">
<span class="def">def</span>
<span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763912496&#39;</span><span class="o">&gt;</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
<span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981979212416&#39;</span><span class="o">&gt;</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
<label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label>
@ -2841,7 +2837,7 @@ is unlikely to come up.</p>
<div class="decorator">@operation(Operation.FROM)</div>
<span class="def">def</span>
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595764027520&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981977951568&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
@ -2910,7 +2906,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@operation(Operation.FROM)</div>
<span class="def">def</span>
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595762701712&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595762701712&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981977374288&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981977374288&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
@ -3115,7 +3111,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@operation(Operation.NO_OP)</div>
<span class="def">def</span>
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595762701712&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595762701712&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981977374288&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981977374288&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
@ -3835,7 +3831,7 @@ and check if it matches the type of the value provided. If not then make it null
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function">
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
@ -3879,7 +3875,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div>
<span class="def">def</span>
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
@ -3900,7 +3896,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div>
<span class="def">def</span>
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span>
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span>
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
@ -3921,7 +3917,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div>
<span class="def">def</span>
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
@ -3948,7 +3944,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div>
<span class="def">def</span>
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
@ -3987,7 +3983,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="attr function">
<span class="def">def</span>
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
@ -4008,7 +4004,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="attr function">
<span class="def">def</span>
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
@ -4602,7 +4598,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function">
<span class="def">def</span>
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
@ -4623,7 +4619,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function">
<span class="def">def</span>
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595766433424&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981981343056&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
@ -4658,7 +4654,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function">
<span class="def">def</span>
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595762334880&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981977905056&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
@ -4903,7 +4899,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="decorator">@classmethod</div>
<span class="def">def</span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
@ -4924,7 +4920,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="decorator">@classmethod</div>
<span class="def">def</span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
@ -5166,7 +5162,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function">
<span class="def">def</span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
@ -5193,7 +5189,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function">
<span class="def">def</span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140595763234272&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;139981978118480&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>

4349
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

3944
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

11
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

1410
docs/sqlglot/dialects/dialect.html
View file

File diff suppressed because it is too large Load diff

7
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

29
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

11
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

3029
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

21
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

11
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

1543
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

33
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

2903
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

534
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

1364
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

700
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

334
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

11
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

3
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

3592
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

3
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

23298
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

18313
docs/sqlglot/generator.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/helper.html
View file

@ -1808,7 +1808,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS">
<div class="attr variable">
<span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;year_month&#39;, &#39;quarter&#39;, &#39;month&#39;, &#39;year&#39;, &#39;week&#39;, &#39;day&#39;}</span>
<span class="default_value">{&#39;quarter&#39;, &#39;month&#39;, &#39;day&#39;, &#39;year&#39;, &#39;week&#39;, &#39;year_month&#39;}</span>
</div>

2
docs/sqlglot/jsonpath.html
View file

@ -578,7 +578,7 @@
<div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;}</span>
</div>

1656
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

356
docs/sqlglot/optimizer/canonicalize.html
View file

@ -117,137 +117,141 @@
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="p">):</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="p">):</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">):</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="ow">not</span> <span class="ow">in</span> <span class="p">(</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">,</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">INTERVAL</span><span class="p">,</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="p">)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">):</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a><span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="p">):</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="p">):</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="p">):</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="ow">not</span> <span class="ow">in</span> <span class="p">(</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">,</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">INTERVAL</span><span class="p">,</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="p">)</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="p">):</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a><span class="c1"># presto has a boolean type whereas tsql doesn&#39;t (people use bits)</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a><span class="c1"># presto has a boolean type whereas tsql doesn&#39;t (people use bits)</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
</span></pre></div>
@ -335,8 +339,13 @@ conversions rely on type inference.</p>
</span><span id="replace_date_funcs-39"><a href="#replace_date_funcs-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="replace_date_funcs-40"><a href="#replace_date_funcs-40"><span class="linenos">40</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
</span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a>
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="replace_date_funcs-47"><a href="#replace_date_funcs-47"><span class="linenos">47</span></a>
</span><span id="replace_date_funcs-48"><a href="#replace_date_funcs-48"><span class="linenos">48</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div>
@ -367,21 +376,21 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#coerce_type"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-60"><a href="#coerce_type-60"><span class="linenos">60</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="coerce_type-61"><a href="#coerce_type-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="coerce_type-62"><a href="#coerce_type-62"><span class="linenos">62</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="coerce_type-63"><a href="#coerce_type-63"><span class="linenos">63</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="coerce_type-64"><a href="#coerce_type-64"><span class="linenos">64</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="coerce_type-65"><a href="#coerce_type-65"><span class="linenos">65</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="coerce_type-66"><a href="#coerce_type-66"><span class="linenos">66</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="coerce_type-67"><a href="#coerce_type-67"><span class="linenos">67</span></a> <span class="p">):</span>
</span><span id="coerce_type-68"><a href="#coerce_type-68"><span class="linenos">68</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="coerce_type-69"><a href="#coerce_type-69"><span class="linenos">69</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a>
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="k">return</span> <span class="n">node</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-65"><a href="#coerce_type-65"><span class="linenos">65</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="coerce_type-66"><a href="#coerce_type-66"><span class="linenos">66</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="coerce_type-67"><a href="#coerce_type-67"><span class="linenos">67</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="coerce_type-68"><a href="#coerce_type-68"><span class="linenos">68</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="coerce_type-69"><a href="#coerce_type-69"><span class="linenos">69</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="p">):</span>
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="coerce_type-75"><a href="#coerce_type-75"><span class="linenos">75</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="coerce_type-76"><a href="#coerce_type-76"><span class="linenos">76</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="coerce_type-77"><a href="#coerce_type-77"><span class="linenos">77</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="coerce_type-78"><a href="#coerce_type-78"><span class="linenos">78</span></a>
</span><span id="coerce_type-79"><a href="#coerce_type-79"><span class="linenos">79</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div>
@ -399,15 +408,14 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#remove_redundant_casts"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-77"><a href="#remove_redundant_casts-77"><span class="linenos">77</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_redundant_casts-78"><a href="#remove_redundant_casts-78"><span class="linenos">78</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="remove_redundant_casts-79"><a href="#remove_redundant_casts-79"><span class="linenos">79</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="remove_redundant_casts-80"><a href="#remove_redundant_casts-80"><span class="linenos">80</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-81"><a href="#remove_redundant_casts-81"><span class="linenos">81</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-82"><a href="#remove_redundant_casts-82"><span class="linenos">82</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-83"><a href="#remove_redundant_casts-83"><span class="linenos">83</span></a> <span class="p">):</span>
</span><span id="remove_redundant_casts-84"><a href="#remove_redundant_casts-84"><span class="linenos">84</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-85"><a href="#remove_redundant_casts-85"><span class="linenos">85</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-82"><a href="#remove_redundant_casts-82"><span class="linenos">82</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_redundant_casts-83"><a href="#remove_redundant_casts-83"><span class="linenos">83</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="remove_redundant_casts-84"><a href="#remove_redundant_casts-84"><span class="linenos">84</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="remove_redundant_casts-85"><a href="#remove_redundant_casts-85"><span class="linenos">85</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-86"><a href="#remove_redundant_casts-86"><span class="linenos">86</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-87"><a href="#remove_redundant_casts-87"><span class="linenos">87</span></a> <span class="p">):</span>
</span><span id="remove_redundant_casts-88"><a href="#remove_redundant_casts-88"><span class="linenos">88</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-89"><a href="#remove_redundant_casts-89"><span class="linenos">89</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -425,23 +433,23 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#ensure_bools"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-88"><a href="#ensure_bools-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="ensure_bools-89"><a href="#ensure_bools-89"><span class="linenos"> 89</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="ensure_bools-90"><a href="#ensure_bools-90"><span class="linenos"> 90</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="ensure_bools-91"><a href="#ensure_bools-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="ensure_bools-92"><a href="#ensure_bools-92"><span class="linenos"> 92</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="ensure_bools-93"><a href="#ensure_bools-93"><span class="linenos"> 93</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="ensure_bools-94"><a href="#ensure_bools-94"><span class="linenos"> 94</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="ensure_bools-95"><a href="#ensure_bools-95"><span class="linenos"> 95</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-96"><a href="#ensure_bools-96"><span class="linenos"> 96</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a> <span class="p">):</span>
</span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a>
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-92"><a href="#ensure_bools-92"><span class="linenos"> 92</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="ensure_bools-93"><a href="#ensure_bools-93"><span class="linenos"> 93</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="ensure_bools-94"><a href="#ensure_bools-94"><span class="linenos"> 94</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="ensure_bools-95"><a href="#ensure_bools-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="ensure_bools-96"><a href="#ensure_bools-96"><span class="linenos"> 96</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a> <span class="p">):</span>
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-105"><a href="#ensure_bools-105"><span class="linenos">105</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="ensure_bools-106"><a href="#ensure_bools-106"><span class="linenos">106</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-107"><a href="#ensure_bools-107"><span class="linenos">107</span></a>
</span><span id="ensure_bools-108"><a href="#ensure_bools-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -459,12 +467,12 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#remove_ascending_order"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-107"><a href="#remove_ascending_order-107"><span class="linenos">107</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_ascending_order-108"><a href="#remove_ascending_order-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="remove_ascending_order-109"><a href="#remove_ascending_order-109"><span class="linenos">109</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="remove_ascending_order-110"><a href="#remove_ascending_order-110"><span class="linenos">110</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="remove_ascending_order-111"><a href="#remove_ascending_order-111"><span class="linenos">111</span></a>
</span><span id="remove_ascending_order-112"><a href="#remove_ascending_order-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-111"><a href="#remove_ascending_order-111"><span class="linenos">111</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_ascending_order-112"><a href="#remove_ascending_order-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="remove_ascending_order-113"><a href="#remove_ascending_order-113"><span class="linenos">113</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="remove_ascending_order-114"><a href="#remove_ascending_order-114"><span class="linenos">114</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="remove_ascending_order-115"><a href="#remove_ascending_order-115"><span class="linenos">115</span></a>
</span><span id="remove_ascending_order-116"><a href="#remove_ascending_order-116"><span class="linenos">116</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>

2
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -585,7 +585,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;match&#39;, &#39;format&#39;, &#39;having&#39;, &#39;windows&#39;, &#39;settings&#39;, &#39;laterals&#39;, &#39;connect&#39;, &#39;qualify&#39;, &#39;group&#39;, &#39;cluster&#39;, &#39;distribute&#39;, &#39;kind&#39;, &#39;sample&#39;, &#39;sort&#39;, &#39;locks&#39;, &#39;offset&#39;, &#39;with&#39;, &#39;pivots&#39;, &#39;into&#39;, &#39;distinct&#39;, &#39;limit&#39;}</span>
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;distinct&#39;, &#39;settings&#39;, &#39;distribute&#39;, &#39;sample&#39;, &#39;pivots&#39;, &#39;laterals&#39;, &#39;into&#39;, &#39;qualify&#39;, &#39;sort&#39;, &#39;cluster&#39;, &#39;limit&#39;, &#39;format&#39;, &#39;connect&#39;, &#39;windows&#39;, &#39;match&#39;, &#39;having&#39;, &#39;offset&#39;, &#39;kind&#39;, &#39;with&#39;, &#39;group&#39;, &#39;locks&#39;}</span>
</div>

251
docs/sqlglot/optimizer/qualify_columns.html
View file

@ -756,67 +756,72 @@
</span><span id="L-662"><a href="#L-662"><span class="linenos">662</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-663"><a href="#L-663"><span class="linenos">663</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="L-664"><a href="#L-664"><span class="linenos">664</span></a>
</span><span id="L-665"><a href="#L-665"><span class="linenos">665</span></a> <span class="c1"># If the source&#39;s columns are aliased, their aliases shadow the corresponding column names</span>
</span><span id="L-666"><a href="#L-666"><span class="linenos">666</span></a> <span class="k">return</span> <span class="p">[</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)]</span>
</span><span id="L-667"><a href="#L-667"><span class="linenos">667</span></a>
</span><span id="L-668"><a href="#L-668"><span class="linenos">668</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
</span><span id="L-669"><a href="#L-669"><span class="linenos">669</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-670"><a href="#L-670"><span class="linenos">670</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-671"><a href="#L-671"><span class="linenos">671</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
</span><span id="L-672"><a href="#L-672"><span class="linenos">672</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
</span><span id="L-673"><a href="#L-673"><span class="linenos">673</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-674"><a href="#L-674"><span class="linenos">674</span></a> <span class="p">)</span>
</span><span id="L-675"><a href="#L-675"><span class="linenos">675</span></a> <span class="p">}</span>
</span><span id="L-676"><a href="#L-676"><span class="linenos">676</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
</span><span id="L-677"><a href="#L-677"><span class="linenos">677</span></a>
</span><span id="L-678"><a href="#L-678"><span class="linenos">678</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
</span><span id="L-679"><a href="#L-679"><span class="linenos">679</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
</span><span id="L-680"><a href="#L-680"><span class="linenos">680</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
</span><span id="L-681"><a href="#L-681"><span class="linenos">681</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-682"><a href="#L-682"><span class="linenos">682</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
</span><span id="L-683"><a href="#L-683"><span class="linenos">683</span></a>
</span><span id="L-684"><a href="#L-684"><span class="linenos">684</span></a><span class="sd"> Args:</span>
</span><span id="L-685"><a href="#L-685"><span class="linenos">685</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
</span><span id="L-686"><a href="#L-686"><span class="linenos">686</span></a>
</span><span id="L-687"><a href="#L-687"><span class="linenos">687</span></a><span class="sd"> Returns:</span>
</span><span id="L-688"><a href="#L-688"><span class="linenos">688</span></a><span class="sd"> Mapping of column name to source name.</span>
</span><span id="L-689"><a href="#L-689"><span class="linenos">689</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-690"><a href="#L-690"><span class="linenos">690</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
</span><span id="L-691"><a href="#L-691"><span class="linenos">691</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-692"><a href="#L-692"><span class="linenos">692</span></a>
</span><span id="L-693"><a href="#L-693"><span class="linenos">693</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
</span><span id="L-694"><a href="#L-694"><span class="linenos">694</span></a>
</span><span id="L-695"><a href="#L-695"><span class="linenos">695</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span><span id="L-696"><a href="#L-696"><span class="linenos">696</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
</span><span id="L-697"><a href="#L-697"><span class="linenos">697</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
</span><span id="L-698"><a href="#L-698"><span class="linenos">698</span></a>
</span><span id="L-699"><a href="#L-699"><span class="linenos">699</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-700"><a href="#L-700"><span class="linenos">700</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-701"><a href="#L-701"><span class="linenos">701</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
</span><span id="L-702"><a href="#L-702"><span class="linenos">702</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-665"><a href="#L-665"><span class="linenos">665</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
</span><span id="L-666"><a href="#L-666"><span class="linenos">666</span></a> <span class="c1"># If the source&#39;s columns are aliased, their aliases shadow the corresponding column names.</span>
</span><span id="L-667"><a href="#L-667"><span class="linenos">667</span></a> <span class="c1"># This can be expensive if there are lots of columns, so only do this if column_aliases exist.</span>
</span><span id="L-668"><a href="#L-668"><span class="linenos">668</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="L-669"><a href="#L-669"><span class="linenos">669</span></a> <span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
</span><span id="L-670"><a href="#L-670"><span class="linenos">670</span></a> <span class="p">]</span>
</span><span id="L-671"><a href="#L-671"><span class="linenos">671</span></a> <span class="k">return</span> <span class="n">columns</span>
</span><span id="L-672"><a href="#L-672"><span class="linenos">672</span></a>
</span><span id="L-673"><a href="#L-673"><span class="linenos">673</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
</span><span id="L-674"><a href="#L-674"><span class="linenos">674</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-675"><a href="#L-675"><span class="linenos">675</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-676"><a href="#L-676"><span class="linenos">676</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
</span><span id="L-677"><a href="#L-677"><span class="linenos">677</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
</span><span id="L-678"><a href="#L-678"><span class="linenos">678</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-679"><a href="#L-679"><span class="linenos">679</span></a> <span class="p">)</span>
</span><span id="L-680"><a href="#L-680"><span class="linenos">680</span></a> <span class="p">}</span>
</span><span id="L-681"><a href="#L-681"><span class="linenos">681</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
</span><span id="L-682"><a href="#L-682"><span class="linenos">682</span></a>
</span><span id="L-683"><a href="#L-683"><span class="linenos">683</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
</span><span id="L-684"><a href="#L-684"><span class="linenos">684</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
</span><span id="L-685"><a href="#L-685"><span class="linenos">685</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
</span><span id="L-686"><a href="#L-686"><span class="linenos">686</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-687"><a href="#L-687"><span class="linenos">687</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
</span><span id="L-688"><a href="#L-688"><span class="linenos">688</span></a>
</span><span id="L-689"><a href="#L-689"><span class="linenos">689</span></a><span class="sd"> Args:</span>
</span><span id="L-690"><a href="#L-690"><span class="linenos">690</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
</span><span id="L-691"><a href="#L-691"><span class="linenos">691</span></a>
</span><span id="L-692"><a href="#L-692"><span class="linenos">692</span></a><span class="sd"> Returns:</span>
</span><span id="L-693"><a href="#L-693"><span class="linenos">693</span></a><span class="sd"> Mapping of column name to source name.</span>
</span><span id="L-694"><a href="#L-694"><span class="linenos">694</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-695"><a href="#L-695"><span class="linenos">695</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
</span><span id="L-696"><a href="#L-696"><span class="linenos">696</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-697"><a href="#L-697"><span class="linenos">697</span></a>
</span><span id="L-698"><a href="#L-698"><span class="linenos">698</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
</span><span id="L-699"><a href="#L-699"><span class="linenos">699</span></a>
</span><span id="L-700"><a href="#L-700"><span class="linenos">700</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span><span id="L-701"><a href="#L-701"><span class="linenos">701</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
</span><span id="L-702"><a href="#L-702"><span class="linenos">702</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
</span><span id="L-703"><a href="#L-703"><span class="linenos">703</span></a>
</span><span id="L-704"><a href="#L-704"><span class="linenos">704</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
</span><span id="L-705"><a href="#L-705"><span class="linenos">705</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-706"><a href="#L-706"><span class="linenos">706</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
</span><span id="L-707"><a href="#L-707"><span class="linenos">707</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-704"><a href="#L-704"><span class="linenos">704</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-705"><a href="#L-705"><span class="linenos">705</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-706"><a href="#L-706"><span class="linenos">706</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
</span><span id="L-707"><a href="#L-707"><span class="linenos">707</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-708"><a href="#L-708"><span class="linenos">708</span></a>
</span><span id="L-709"><a href="#L-709"><span class="linenos">709</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
</span><span id="L-710"><a href="#L-710"><span class="linenos">710</span></a>
</span><span id="L-711"><a href="#L-711"><span class="linenos">711</span></a> <span class="nd">@staticmethod</span>
</span><span id="L-712"><a href="#L-712"><span class="linenos">712</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
</span><span id="L-713"><a href="#L-713"><span class="linenos">713</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-714"><a href="#L-714"><span class="linenos">714</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
</span><span id="L-709"><a href="#L-709"><span class="linenos">709</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
</span><span id="L-710"><a href="#L-710"><span class="linenos">710</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-711"><a href="#L-711"><span class="linenos">711</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
</span><span id="L-712"><a href="#L-712"><span class="linenos">712</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-713"><a href="#L-713"><span class="linenos">713</span></a>
</span><span id="L-714"><a href="#L-714"><span class="linenos">714</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
</span><span id="L-715"><a href="#L-715"><span class="linenos">715</span></a>
</span><span id="L-716"><a href="#L-716"><span class="linenos">716</span></a><span class="sd"> Example:</span>
</span><span id="L-717"><a href="#L-717"><span class="linenos">717</span></a><span class="sd"> &gt;&gt;&gt; sorted(Resolver._find_unique_columns([&quot;a&quot;, &quot;b&quot;, &quot;b&quot;, &quot;c&quot;]))</span>
</span><span id="L-718"><a href="#L-718"><span class="linenos">718</span></a><span class="sd"> [&#39;a&#39;, &#39;c&#39;]</span>
</span><span id="L-719"><a href="#L-719"><span class="linenos">719</span></a>
</span><span id="L-720"><a href="#L-720"><span class="linenos">720</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
</span><span id="L-721"><a href="#L-721"><span class="linenos">721</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-722"><a href="#L-722"><span class="linenos">722</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-723"><a href="#L-723"><span class="linenos">723</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
</span><span id="L-724"><a href="#L-724"><span class="linenos">724</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
</span><span id="L-725"><a href="#L-725"><span class="linenos">725</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
</span><span id="L-716"><a href="#L-716"><span class="linenos">716</span></a> <span class="nd">@staticmethod</span>
</span><span id="L-717"><a href="#L-717"><span class="linenos">717</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
</span><span id="L-718"><a href="#L-718"><span class="linenos">718</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-719"><a href="#L-719"><span class="linenos">719</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
</span><span id="L-720"><a href="#L-720"><span class="linenos">720</span></a>
</span><span id="L-721"><a href="#L-721"><span class="linenos">721</span></a><span class="sd"> Example:</span>
</span><span id="L-722"><a href="#L-722"><span class="linenos">722</span></a><span class="sd"> &gt;&gt;&gt; sorted(Resolver._find_unique_columns([&quot;a&quot;, &quot;b&quot;, &quot;b&quot;, &quot;c&quot;]))</span>
</span><span id="L-723"><a href="#L-723"><span class="linenos">723</span></a><span class="sd"> [&#39;a&#39;, &#39;c&#39;]</span>
</span><span id="L-724"><a href="#L-724"><span class="linenos">724</span></a>
</span><span id="L-725"><a href="#L-725"><span class="linenos">725</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
</span><span id="L-726"><a href="#L-726"><span class="linenos">726</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-727"><a href="#L-727"><span class="linenos">727</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-728"><a href="#L-728"><span class="linenos">728</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
</span><span id="L-729"><a href="#L-729"><span class="linenos">729</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
</span><span id="L-730"><a href="#L-730"><span class="linenos">730</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
</span></pre></div>
@ -1228,67 +1233,72 @@ know what you're doing!</li>
</span><span id="Resolver-663"><a href="#Resolver-663"><span class="linenos">663</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="Resolver-664"><a href="#Resolver-664"><span class="linenos">664</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="Resolver-665"><a href="#Resolver-665"><span class="linenos">665</span></a>
</span><span id="Resolver-666"><a href="#Resolver-666"><span class="linenos">666</span></a> <span class="c1"># If the source&#39;s columns are aliased, their aliases shadow the corresponding column names</span>
</span><span id="Resolver-667"><a href="#Resolver-667"><span class="linenos">667</span></a> <span class="k">return</span> <span class="p">[</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)]</span>
</span><span id="Resolver-668"><a href="#Resolver-668"><span class="linenos">668</span></a>
</span><span id="Resolver-669"><a href="#Resolver-669"><span class="linenos">669</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
</span><span id="Resolver-670"><a href="#Resolver-670"><span class="linenos">670</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="Resolver-671"><a href="#Resolver-671"><span class="linenos">671</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="Resolver-672"><a href="#Resolver-672"><span class="linenos">672</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
</span><span id="Resolver-673"><a href="#Resolver-673"><span class="linenos">673</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
</span><span id="Resolver-674"><a href="#Resolver-674"><span class="linenos">674</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="Resolver-675"><a href="#Resolver-675"><span class="linenos">675</span></a> <span class="p">)</span>
</span><span id="Resolver-676"><a href="#Resolver-676"><span class="linenos">676</span></a> <span class="p">}</span>
</span><span id="Resolver-677"><a href="#Resolver-677"><span class="linenos">677</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
</span><span id="Resolver-678"><a href="#Resolver-678"><span class="linenos">678</span></a>
</span><span id="Resolver-679"><a href="#Resolver-679"><span class="linenos">679</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
</span><span id="Resolver-680"><a href="#Resolver-680"><span class="linenos">680</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
</span><span id="Resolver-681"><a href="#Resolver-681"><span class="linenos">681</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
</span><span id="Resolver-682"><a href="#Resolver-682"><span class="linenos">682</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="Resolver-683"><a href="#Resolver-683"><span class="linenos">683</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
</span><span id="Resolver-684"><a href="#Resolver-684"><span class="linenos">684</span></a>
</span><span id="Resolver-685"><a href="#Resolver-685"><span class="linenos">685</span></a><span class="sd"> Args:</span>
</span><span id="Resolver-686"><a href="#Resolver-686"><span class="linenos">686</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
</span><span id="Resolver-687"><a href="#Resolver-687"><span class="linenos">687</span></a>
</span><span id="Resolver-688"><a href="#Resolver-688"><span class="linenos">688</span></a><span class="sd"> Returns:</span>
</span><span id="Resolver-689"><a href="#Resolver-689"><span class="linenos">689</span></a><span class="sd"> Mapping of column name to source name.</span>
</span><span id="Resolver-690"><a href="#Resolver-690"><span class="linenos">690</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="Resolver-691"><a href="#Resolver-691"><span class="linenos">691</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
</span><span id="Resolver-692"><a href="#Resolver-692"><span class="linenos">692</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="Resolver-693"><a href="#Resolver-693"><span class="linenos">693</span></a>
</span><span id="Resolver-694"><a href="#Resolver-694"><span class="linenos">694</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
</span><span id="Resolver-695"><a href="#Resolver-695"><span class="linenos">695</span></a>
</span><span id="Resolver-696"><a href="#Resolver-696"><span class="linenos">696</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span><span id="Resolver-697"><a href="#Resolver-697"><span class="linenos">697</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
</span><span id="Resolver-698"><a href="#Resolver-698"><span class="linenos">698</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
</span><span id="Resolver-699"><a href="#Resolver-699"><span class="linenos">699</span></a>
</span><span id="Resolver-700"><a href="#Resolver-700"><span class="linenos">700</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="Resolver-701"><a href="#Resolver-701"><span class="linenos">701</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Resolver-702"><a href="#Resolver-702"><span class="linenos">702</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
</span><span id="Resolver-703"><a href="#Resolver-703"><span class="linenos">703</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Resolver-666"><a href="#Resolver-666"><span class="linenos">666</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
</span><span id="Resolver-667"><a href="#Resolver-667"><span class="linenos">667</span></a> <span class="c1"># If the source&#39;s columns are aliased, their aliases shadow the corresponding column names.</span>
</span><span id="Resolver-668"><a href="#Resolver-668"><span class="linenos">668</span></a> <span class="c1"># This can be expensive if there are lots of columns, so only do this if column_aliases exist.</span>
</span><span id="Resolver-669"><a href="#Resolver-669"><span class="linenos">669</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="Resolver-670"><a href="#Resolver-670"><span class="linenos">670</span></a> <span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
</span><span id="Resolver-671"><a href="#Resolver-671"><span class="linenos">671</span></a> <span class="p">]</span>
</span><span id="Resolver-672"><a href="#Resolver-672"><span class="linenos">672</span></a> <span class="k">return</span> <span class="n">columns</span>
</span><span id="Resolver-673"><a href="#Resolver-673"><span class="linenos">673</span></a>
</span><span id="Resolver-674"><a href="#Resolver-674"><span class="linenos">674</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
</span><span id="Resolver-675"><a href="#Resolver-675"><span class="linenos">675</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="Resolver-676"><a href="#Resolver-676"><span class="linenos">676</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="Resolver-677"><a href="#Resolver-677"><span class="linenos">677</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
</span><span id="Resolver-678"><a href="#Resolver-678"><span class="linenos">678</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
</span><span id="Resolver-679"><a href="#Resolver-679"><span class="linenos">679</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="Resolver-680"><a href="#Resolver-680"><span class="linenos">680</span></a> <span class="p">)</span>
</span><span id="Resolver-681"><a href="#Resolver-681"><span class="linenos">681</span></a> <span class="p">}</span>
</span><span id="Resolver-682"><a href="#Resolver-682"><span class="linenos">682</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
</span><span id="Resolver-683"><a href="#Resolver-683"><span class="linenos">683</span></a>
</span><span id="Resolver-684"><a href="#Resolver-684"><span class="linenos">684</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
</span><span id="Resolver-685"><a href="#Resolver-685"><span class="linenos">685</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
</span><span id="Resolver-686"><a href="#Resolver-686"><span class="linenos">686</span></a> <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
</span><span id="Resolver-687"><a href="#Resolver-687"><span class="linenos">687</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="Resolver-688"><a href="#Resolver-688"><span class="linenos">688</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
</span><span id="Resolver-689"><a href="#Resolver-689"><span class="linenos">689</span></a>
</span><span id="Resolver-690"><a href="#Resolver-690"><span class="linenos">690</span></a><span class="sd"> Args:</span>
</span><span id="Resolver-691"><a href="#Resolver-691"><span class="linenos">691</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
</span><span id="Resolver-692"><a href="#Resolver-692"><span class="linenos">692</span></a>
</span><span id="Resolver-693"><a href="#Resolver-693"><span class="linenos">693</span></a><span class="sd"> Returns:</span>
</span><span id="Resolver-694"><a href="#Resolver-694"><span class="linenos">694</span></a><span class="sd"> Mapping of column name to source name.</span>
</span><span id="Resolver-695"><a href="#Resolver-695"><span class="linenos">695</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="Resolver-696"><a href="#Resolver-696"><span class="linenos">696</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
</span><span id="Resolver-697"><a href="#Resolver-697"><span class="linenos">697</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="Resolver-698"><a href="#Resolver-698"><span class="linenos">698</span></a>
</span><span id="Resolver-699"><a href="#Resolver-699"><span class="linenos">699</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
</span><span id="Resolver-700"><a href="#Resolver-700"><span class="linenos">700</span></a>
</span><span id="Resolver-701"><a href="#Resolver-701"><span class="linenos">701</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
</span><span id="Resolver-702"><a href="#Resolver-702"><span class="linenos">702</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
</span><span id="Resolver-703"><a href="#Resolver-703"><span class="linenos">703</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
</span><span id="Resolver-704"><a href="#Resolver-704"><span class="linenos">704</span></a>
</span><span id="Resolver-705"><a href="#Resolver-705"><span class="linenos">705</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
</span><span id="Resolver-706"><a href="#Resolver-706"><span class="linenos">706</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="Resolver-707"><a href="#Resolver-707"><span class="linenos">707</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
</span><span id="Resolver-708"><a href="#Resolver-708"><span class="linenos">708</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="Resolver-705"><a href="#Resolver-705"><span class="linenos">705</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="Resolver-706"><a href="#Resolver-706"><span class="linenos">706</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Resolver-707"><a href="#Resolver-707"><span class="linenos">707</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
</span><span id="Resolver-708"><a href="#Resolver-708"><span class="linenos">708</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Resolver-709"><a href="#Resolver-709"><span class="linenos">709</span></a>
</span><span id="Resolver-710"><a href="#Resolver-710"><span class="linenos">710</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
</span><span id="Resolver-711"><a href="#Resolver-711"><span class="linenos">711</span></a>
</span><span id="Resolver-712"><a href="#Resolver-712"><span class="linenos">712</span></a> <span class="nd">@staticmethod</span>
</span><span id="Resolver-713"><a href="#Resolver-713"><span class="linenos">713</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
</span><span id="Resolver-714"><a href="#Resolver-714"><span class="linenos">714</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="Resolver-715"><a href="#Resolver-715"><span class="linenos">715</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
</span><span id="Resolver-710"><a href="#Resolver-710"><span class="linenos">710</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
</span><span id="Resolver-711"><a href="#Resolver-711"><span class="linenos">711</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="Resolver-712"><a href="#Resolver-712"><span class="linenos">712</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
</span><span id="Resolver-713"><a href="#Resolver-713"><span class="linenos">713</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="Resolver-714"><a href="#Resolver-714"><span class="linenos">714</span></a>
</span><span id="Resolver-715"><a href="#Resolver-715"><span class="linenos">715</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
</span><span id="Resolver-716"><a href="#Resolver-716"><span class="linenos">716</span></a>
</span><span id="Resolver-717"><a href="#Resolver-717"><span class="linenos">717</span></a><span class="sd"> Example:</span>
</span><span id="Resolver-718"><a href="#Resolver-718"><span class="linenos">718</span></a><span class="sd"> &gt;&gt;&gt; sorted(Resolver._find_unique_columns([&quot;a&quot;, &quot;b&quot;, &quot;b&quot;, &quot;c&quot;]))</span>
</span><span id="Resolver-719"><a href="#Resolver-719"><span class="linenos">719</span></a><span class="sd"> [&#39;a&#39;, &#39;c&#39;]</span>
</span><span id="Resolver-720"><a href="#Resolver-720"><span class="linenos">720</span></a>
</span><span id="Resolver-721"><a href="#Resolver-721"><span class="linenos">721</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
</span><span id="Resolver-722"><a href="#Resolver-722"><span class="linenos">722</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="Resolver-723"><a href="#Resolver-723"><span class="linenos">723</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="Resolver-724"><a href="#Resolver-724"><span class="linenos">724</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
</span><span id="Resolver-725"><a href="#Resolver-725"><span class="linenos">725</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
</span><span id="Resolver-726"><a href="#Resolver-726"><span class="linenos">726</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
</span><span id="Resolver-717"><a href="#Resolver-717"><span class="linenos">717</span></a> <span class="nd">@staticmethod</span>
</span><span id="Resolver-718"><a href="#Resolver-718"><span class="linenos">718</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
</span><span id="Resolver-719"><a href="#Resolver-719"><span class="linenos">719</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="Resolver-720"><a href="#Resolver-720"><span class="linenos">720</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
</span><span id="Resolver-721"><a href="#Resolver-721"><span class="linenos">721</span></a>
</span><span id="Resolver-722"><a href="#Resolver-722"><span class="linenos">722</span></a><span class="sd"> Example:</span>
</span><span id="Resolver-723"><a href="#Resolver-723"><span class="linenos">723</span></a><span class="sd"> &gt;&gt;&gt; sorted(Resolver._find_unique_columns([&quot;a&quot;, &quot;b&quot;, &quot;b&quot;, &quot;c&quot;]))</span>
</span><span id="Resolver-724"><a href="#Resolver-724"><span class="linenos">724</span></a><span class="sd"> [&#39;a&#39;, &#39;c&#39;]</span>
</span><span id="Resolver-725"><a href="#Resolver-725"><span class="linenos">725</span></a>
</span><span id="Resolver-726"><a href="#Resolver-726"><span class="linenos">726</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
</span><span id="Resolver-727"><a href="#Resolver-727"><span class="linenos">727</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="Resolver-728"><a href="#Resolver-728"><span class="linenos">728</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="Resolver-729"><a href="#Resolver-729"><span class="linenos">729</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
</span><span id="Resolver-730"><a href="#Resolver-730"><span class="linenos">730</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
</span><span id="Resolver-731"><a href="#Resolver-731"><span class="linenos">731</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
</span></pre></div>
@ -1471,8 +1481,13 @@ know what you're doing!</li>
</span><span id="Resolver.get_source_columns-663"><a href="#Resolver.get_source_columns-663"><span class="linenos">663</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="Resolver.get_source_columns-664"><a href="#Resolver.get_source_columns-664"><span class="linenos">664</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="Resolver.get_source_columns-665"><a href="#Resolver.get_source_columns-665"><span class="linenos">665</span></a>
</span><span id="Resolver.get_source_columns-666"><a href="#Resolver.get_source_columns-666"><span class="linenos">666</span></a> <span class="c1"># If the source&#39;s columns are aliased, their aliases shadow the corresponding column names</span>
</span><span id="Resolver.get_source_columns-667"><a href="#Resolver.get_source_columns-667"><span class="linenos">667</span></a> <span class="k">return</span> <span class="p">[</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)]</span>
</span><span id="Resolver.get_source_columns-666"><a href="#Resolver.get_source_columns-666"><span class="linenos">666</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
</span><span id="Resolver.get_source_columns-667"><a href="#Resolver.get_source_columns-667"><span class="linenos">667</span></a> <span class="c1"># If the source&#39;s columns are aliased, their aliases shadow the corresponding column names.</span>
</span><span id="Resolver.get_source_columns-668"><a href="#Resolver.get_source_columns-668"><span class="linenos">668</span></a> <span class="c1"># This can be expensive if there are lots of columns, so only do this if column_aliases exist.</span>
</span><span id="Resolver.get_source_columns-669"><a href="#Resolver.get_source_columns-669"><span class="linenos">669</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="Resolver.get_source_columns-670"><a href="#Resolver.get_source_columns-670"><span class="linenos">670</span></a> <span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
</span><span id="Resolver.get_source_columns-671"><a href="#Resolver.get_source_columns-671"><span class="linenos">671</span></a> <span class="p">]</span>
</span><span id="Resolver.get_source_columns-672"><a href="#Resolver.get_source_columns-672"><span class="linenos">672</span></a> <span class="k">return</span> <span class="n">columns</span>
</span></pre></div>

4
docs/sqlglot/optimizer/simplify.html
View file

@ -2546,7 +2546,7 @@ prefix are statically known.</p>
<div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;}</span>
</div>
@ -2626,7 +2626,7 @@ prefix are statically known.</p>
<section id="JOINS">
<div class="attr variable">
<span class="name">JOINS</span> =
<span class="default_value">{(&#39;&#39;, &#39;&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;)}</span>
<span class="default_value">{(&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;&#39;, &#39;&#39;)}</span>
</div>

21006
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

5210
docs/sqlglot/tokens.html
View file

File diff suppressed because it is too large Load diff

1374
docs/sqlglot/transforms.html
View file

File diff suppressed because it is too large Load diff

2
sqlglot/dataframe/sql/functions.py
View file

@ -148,7 +148,7 @@ def atanh(col: ColumnOrName) -> Column:
def cbrt(col: ColumnOrName) -> Column:
return Column.invoke_anonymous_function(col, "CBRT")
return Column.invoke_expression_over_column(col, expression.Cbrt)
def ceil(col: ColumnOrName) -> Column:

10
sqlglot/dataframe/sql/session.py
View file

@ -70,12 +70,10 @@ class SparkSession:
column_mapping = {f"_{i}": None for i in range(1, len(data[0]) + 1)}
data_expressions = [
exp.Tuple(
expressions=list(
map(
lambda x: F.lit(x).expression,
row if not isinstance(row, dict) else row.values(),
)
exp.tuple_(
*map(
lambda x: F.lit(x).expression,
row if not isinstance(row, dict) else row.values(),
)
)
for row in data

50
sqlglot/dialects/bigquery.py
View file

@ -39,24 +39,31 @@ def _derived_table_values_to_unnest(self: BigQuery.Generator, expression: exp.Va
alias = expression.args.get("alias")
structs = [
exp.Struct(
return self.unnest_sql(
exp.Unnest(
expressions=[
exp.alias_(value, column_name)
for value, column_name in zip(
t.expressions,
(
alias.columns
if alias and alias.columns
else (f"_c{i}" for i in range(len(t.expressions)))
exp.array(
*(
exp.Struct(
expressions=[
exp.alias_(value, column_name)
for value, column_name in zip(
t.expressions,
(
alias.columns
if alias and alias.columns
else (f"_c{i}" for i in range(len(t.expressions)))
),
)
]
)
for t in expression.find_all(exp.Tuple)
),
copy=False,
)
]
)
for t in expression.find_all(exp.Tuple)
]
return self.unnest_sql(exp.Unnest(expressions=[exp.Array(expressions=structs)]))
)
def _returnsproperty_sql(self: BigQuery.Generator, expression: exp.ReturnsProperty) -> str:
@ -161,12 +168,18 @@ def _pushdown_cte_column_names(expression: exp.Expression) -> exp.Expression:
return expression
def _parse_timestamp(args: t.List) -> exp.StrToTime:
def _parse_parse_timestamp(args: t.List) -> exp.StrToTime:
this = format_time_lambda(exp.StrToTime, "bigquery")([seq_get(args, 1), seq_get(args, 0)])
this.set("zone", seq_get(args, 2))
return this
def _parse_timestamp(args: t.List) -> exp.Timestamp:
timestamp = exp.Timestamp.from_arg_list(args)
timestamp.set("with_tz", True)
return timestamp
def _parse_date(args: t.List) -> exp.Date | exp.DateFromParts:
expr_type = exp.DateFromParts if len(args) == 3 else exp.Date
return expr_type.from_arg_list(args)
@ -318,6 +331,7 @@ class BigQuery(Dialect):
"TIMESTAMP": TokenType.TIMESTAMPTZ,
}
KEYWORDS.pop("DIV")
KEYWORDS.pop("VALUES")
class Parser(parser.Parser):
PREFIXED_PIVOT_COLUMNS = True
@ -348,7 +362,7 @@ class BigQuery(Dialect):
"PARSE_DATE": lambda args: format_time_lambda(exp.StrToDate, "bigquery")(
[seq_get(args, 1), seq_get(args, 0)]
),
"PARSE_TIMESTAMP": _parse_timestamp,
"PARSE_TIMESTAMP": _parse_parse_timestamp,
"REGEXP_CONTAINS": exp.RegexpLike.from_arg_list,
"REGEXP_EXTRACT": lambda args: exp.RegexpExtract(
this=seq_get(args, 0),
@ -367,6 +381,7 @@ class BigQuery(Dialect):
"TIME": _parse_time,
"TIME_ADD": parse_date_delta_with_interval(exp.TimeAdd),
"TIME_SUB": parse_date_delta_with_interval(exp.TimeSub),
"TIMESTAMP": _parse_timestamp,
"TIMESTAMP_ADD": parse_date_delta_with_interval(exp.TimestampAdd),
"TIMESTAMP_SUB": parse_date_delta_with_interval(exp.TimestampSub),
"TIMESTAMP_MICROS": lambda args: exp.UnixToTime(
@ -395,11 +410,6 @@ class BigQuery(Dialect):
TokenType.TABLE,
}
ID_VAR_TOKENS = {
*parser.Parser.ID_VAR_TOKENS,
TokenType.VALUES,
}
PROPERTY_PARSERS = {
**parser.Parser.PROPERTY_PARSERS,
"NOT DETERMINISTIC": lambda self: self.expression(

1
sqlglot/dialects/clickhouse.py
View file

@ -93,6 +93,7 @@ class ClickHouse(Dialect):
"IPV6": TokenType.IPV6,
"AGGREGATEFUNCTION": TokenType.AGGREGATEFUNCTION,
"SIMPLEAGGREGATEFUNCTION": TokenType.SIMPLEAGGREGATEFUNCTION,
"SYSTEM": TokenType.COMMAND,
}
SINGLE_TOKENS = {

29
sqlglot/dialects/dialect.py
View file

@ -654,28 +654,6 @@ def time_format(
return _time_format
def create_with_partitions_sql(self: Generator, expression: exp.Create) -> str:
"""
In Hive and Spark, the PARTITIONED BY property acts as an extension of a table's schema. When the
PARTITIONED BY value is an array of column names, they are transformed into a schema. The corresponding
columns are removed from the create statement.
"""
has_schema = isinstance(expression.this, exp.Schema)
is_partitionable = expression.args.get("kind") in ("TABLE", "VIEW")
if has_schema and is_partitionable:
prop = expression.find(exp.PartitionedByProperty)
if prop and prop.this and not isinstance(prop.this, exp.Schema):
schema = expression.this
columns = {v.name.upper() for v in prop.this.expressions}
partitions = [col for col in schema.expressions if col.name.upper() in columns]
schema.set("expressions", [e for e in schema.expressions if e not in partitions])
prop.replace(exp.PartitionedByProperty(this=exp.Schema(expressions=partitions)))
expression.set("this", schema)
return self.create_sql(expression)
def parse_date_delta(
exp_class: t.Type[E], unit_mapping: t.Optional[t.Dict[str, str]] = None
) -> t.Callable[[t.List], E]:
@ -742,7 +720,10 @@ def timestamptrunc_sql(self: Generator, expression: exp.TimestampTrunc) -> str:
def no_timestamp_sql(self: Generator, expression: exp.Timestamp) -> str:
if not expression.expression:
return self.sql(exp.cast(expression.this, to=exp.DataType.Type.TIMESTAMP))
from sqlglot.optimizer.annotate_types import annotate_types
target_type = annotate_types(expression).type or exp.DataType.Type.TIMESTAMP
return self.sql(exp.cast(expression.this, to=target_type))
if expression.text("expression").lower() in TIMEZONES:
return self.sql(
exp.AtTimeZone(
@ -750,7 +731,7 @@ def no_timestamp_sql(self: Generator, expression: exp.Timestamp) -> str:
zone=expression.expression,
)
)
return self.function_fallback_sql(expression)
return self.func("TIMESTAMP", expression.this, expression.expression)
def locate_to_strposition(args: t.List) -> exp.Expression:

4
sqlglot/dialects/drill.py
View file

@ -5,7 +5,6 @@ import typing as t
from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import (
Dialect,
create_with_partitions_sql,
datestrtodate_sql,
format_time_lambda,
no_trycast_sql,
@ -13,6 +12,7 @@ from sqlglot.dialects.dialect import (
str_position_sql,
timestrtotime_sql,
)
from sqlglot.transforms import preprocess, move_schema_columns_to_partitioned_by
def _date_add_sql(kind: str) -> t.Callable[[Drill.Generator, exp.DateAdd | exp.DateSub], str]:
@ -125,7 +125,7 @@ class Drill(Dialect):
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
exp.ArrayContains: rename_func("REPEATED_CONTAINS"),
exp.ArraySize: rename_func("REPEATED_COUNT"),
exp.Create: create_with_partitions_sql,
exp.Create: preprocess([move_schema_columns_to_partitioned_by]),
exp.DateAdd: _date_add_sql("ADD"),
exp.DateStrToDate: datestrtodate_sql,
exp.DateSub: _date_add_sql("SUB"),

44
sqlglot/dialects/hive.py
View file

@ -9,7 +9,6 @@ from sqlglot.dialects.dialect import (
NormalizationStrategy,
approx_count_distinct_sql,
arg_max_or_min_no_count,
create_with_partitions_sql,
datestrtodate_sql,
format_time_lambda,
if_sql,
@ -32,6 +31,12 @@ from sqlglot.dialects.dialect import (
timestrtotime_sql,
var_map_sql,
)
from sqlglot.transforms import (
remove_unique_constraints,
ctas_with_tmp_tables_to_create_tmp_view,
preprocess,
move_schema_columns_to_partitioned_by,
)
from sqlglot.helper import seq_get
from sqlglot.parser import parse_var_map
from sqlglot.tokens import TokenType
@ -55,30 +60,6 @@ TIME_DIFF_FACTOR = {
DIFF_MONTH_SWITCH = ("YEAR", "QUARTER", "MONTH")
def _create_sql(self, expression: exp.Create) -> str:
# remove UNIQUE column constraints
for constraint in expression.find_all(exp.UniqueColumnConstraint):
if constraint.parent:
constraint.parent.pop()
properties = expression.args.get("properties")
temporary = any(
isinstance(prop, exp.TemporaryProperty)
for prop in (properties.expressions if properties else [])
)
# CTAS with temp tables map to CREATE TEMPORARY VIEW
kind = expression.args["kind"]
if kind.upper() == "TABLE" and temporary:
if expression.expression:
return f"CREATE TEMPORARY VIEW {self.sql(expression, 'this')} AS {self.sql(expression, 'expression')}"
else:
# CREATE TEMPORARY TABLE may require storage provider
expression = self.temporary_storage_provider(expression)
return create_with_partitions_sql(self, expression)
def _add_date_sql(self: Hive.Generator, expression: DATE_ADD_OR_SUB) -> str:
if isinstance(expression, exp.TsOrDsAdd) and not expression.unit:
return self.func("DATE_ADD", expression.this, expression.expression)
@ -285,6 +266,7 @@ class Hive(Dialect):
class Parser(parser.Parser):
LOG_DEFAULTS_TO_LN = True
STRICT_CAST = False
VALUES_FOLLOWED_BY_PAREN = False
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
@ -518,7 +500,13 @@ class Hive(Dialect):
"" if e.args.get("allow_null") else "NOT NULL"
),
exp.VarMap: var_map_sql,
exp.Create: _create_sql,
exp.Create: preprocess(
[
remove_unique_constraints,
ctas_with_tmp_tables_to_create_tmp_view,
move_schema_columns_to_partitioned_by,
]
),
exp.Quantile: rename_func("PERCENTILE"),
exp.ApproxQuantile: rename_func("PERCENTILE_APPROX"),
exp.RegexpExtract: regexp_extract_sql,
@ -581,10 +569,6 @@ class Hive(Dialect):
return super()._jsonpathkey_sql(expression)
def temporary_storage_provider(self, expression: exp.Create) -> exp.Create:
# Hive has no temporary storage provider (there are hive settings though)
return expression
def parameter_sql(self, expression: exp.Parameter) -> str:
this = self.sql(expression, "this")
expression_sql = self.sql(expression, "expression")

1
sqlglot/dialects/mysql.py
View file

@ -445,6 +445,7 @@ class MySQL(Dialect):
LOG_DEFAULTS_TO_LN = True
STRING_ALIASES = True
VALUES_FOLLOWED_BY_PAREN = False
def _parse_primary_key_part(self) -> t.Optional[exp.Expression]:
this = self._parse_id_var()

1
sqlglot/dialects/oracle.py
View file

@ -88,6 +88,7 @@ class Oracle(Dialect):
class Parser(parser.Parser):
ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = False
WINDOW_BEFORE_PAREN_TOKENS = {TokenType.OVER, TokenType.KEEP}
VALUES_FOLLOWED_BY_PAREN = False
FUNCTIONS = {
**parser.Parser.FUNCTIONS,

2
sqlglot/dialects/postgres.py
View file

@ -244,6 +244,8 @@ class Postgres(Dialect):
"@@": TokenType.DAT,
"@>": TokenType.AT_GT,
"<@": TokenType.LT_AT,
"|/": TokenType.PIPE_SLASH,
"||/": TokenType.DPIPE_SLASH,
"BEGIN": TokenType.COMMAND,
"BEGIN TRANSACTION": TokenType.BEGIN,
"BIGSERIAL": TokenType.BIGSERIAL,

2
sqlglot/dialects/presto.py
View file

@ -225,6 +225,8 @@ class Presto(Dialect):
}
class Parser(parser.Parser):
VALUES_FOLLOWED_BY_PAREN = False
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"ARBITRARY": exp.AnyValue.from_arg_list,

7
sqlglot/dialects/redshift.py
View file

@ -136,11 +136,11 @@ class Redshift(Postgres):
refs.add(
(
this.args["from"] if i == 0 else this.args["joins"][i - 1]
).alias_or_name.lower()
).this.alias.lower()
)
table = join.this
if isinstance(table, exp.Table):
table = join.this
if isinstance(table, exp.Table) and not join.args.get("on"):
if table.parts[0].name.lower() in refs:
table.replace(table.to_column())
return this
@ -158,6 +158,7 @@ class Redshift(Postgres):
"UNLOAD": TokenType.COMMAND,
"VARBYTE": TokenType.VARBINARY,
}
KEYWORDS.pop("VALUES")
# Redshift allows # to appear as a table identifier prefix
SINGLE_TOKENS = Postgres.Tokenizer.SINGLE_TOKENS.copy()

2
sqlglot/dialects/snowflake.py
View file

@ -477,6 +477,8 @@ class Snowflake(Dialect):
"PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
"TERSE PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
"COLUMNS": _show_parser("COLUMNS"),
"USERS": _show_parser("USERS"),
"TERSE USERS": _show_parser("USERS"),
}
STAGED_FILE_SINGLE_TOKENS = {

28
sqlglot/dialects/spark.py
View file

@ -5,8 +5,14 @@ import typing as t
from sqlglot import exp
from sqlglot.dialects.dialect import rename_func
from sqlglot.dialects.hive import _parse_ignore_nulls
from sqlglot.dialects.spark2 import Spark2
from sqlglot.dialects.spark2 import Spark2, temporary_storage_provider
from sqlglot.helper import seq_get
from sqlglot.transforms import (
ctas_with_tmp_tables_to_create_tmp_view,
remove_unique_constraints,
preprocess,
move_partitioned_by_to_schema_columns,
)
def _parse_datediff(args: t.List) -> exp.Expression:
@ -35,6 +41,15 @@ def _parse_datediff(args: t.List) -> exp.Expression:
)
def _normalize_partition(e: exp.Expression) -> exp.Expression:
"""Normalize the expressions in PARTITION BY (<expression>, <expression>, ...)"""
if isinstance(e, str):
return exp.to_identifier(e)
if isinstance(e, exp.Literal):
return exp.to_identifier(e.name)
return e
class Spark(Spark2):
class Tokenizer(Spark2.Tokenizer):
RAW_STRINGS = [
@ -72,6 +87,17 @@ class Spark(Spark2):
TRANSFORMS = {
**Spark2.Generator.TRANSFORMS,
exp.Create: preprocess(
[
remove_unique_constraints,
lambda e: ctas_with_tmp_tables_to_create_tmp_view(
e, temporary_storage_provider
),
move_partitioned_by_to_schema_columns,
]
),
exp.PartitionedByProperty: lambda self,
e: f"PARTITIONED BY {self.wrap(self.expressions(sqls=[_normalize_partition(e) for e in e.this.expressions], skip_first=True))}",
exp.StartsWith: rename_func("STARTSWITH"),
exp.TimestampAdd: lambda self, e: self.func(
"DATEADD", e.args.get("unit") or "DAY", e.expression, e.this

29
sqlglot/dialects/spark2.py
View file

@ -13,6 +13,12 @@ from sqlglot.dialects.dialect import (
)
from sqlglot.dialects.hive import Hive
from sqlglot.helper import seq_get
from sqlglot.transforms import (
preprocess,
remove_unique_constraints,
ctas_with_tmp_tables_to_create_tmp_view,
move_schema_columns_to_partitioned_by,
)
def _map_sql(self: Spark2.Generator, expression: exp.Map) -> str:
@ -95,6 +101,13 @@ def _unqualify_pivot_columns(expression: exp.Expression) -> exp.Expression:
return expression
def temporary_storage_provider(expression: exp.Expression) -> exp.Expression:
# spark2, spark, Databricks require a storage provider for temporary tables
provider = exp.FileFormatProperty(this=exp.Literal.string("parquet"))
expression.args["properties"].append("expressions", provider)
return expression
class Spark2(Hive):
class Parser(Hive.Parser):
TRIM_PATTERN_FIRST = True
@ -121,7 +134,6 @@ class Spark2(Hive):
),
zone=seq_get(args, 1),
),
"IIF": exp.If.from_arg_list,
"INT": _parse_as_cast("int"),
"MAP_FROM_ARRAYS": exp.Map.from_arg_list,
"RLIKE": exp.RegexpLike.from_arg_list,
@ -193,6 +205,15 @@ class Spark2(Hive):
e: f"FROM_UTC_TIMESTAMP({self.sql(e, 'this')}, {self.sql(e, 'zone')})",
exp.BitwiseLeftShift: rename_func("SHIFTLEFT"),
exp.BitwiseRightShift: rename_func("SHIFTRIGHT"),
exp.Create: preprocess(
[
remove_unique_constraints,
lambda e: ctas_with_tmp_tables_to_create_tmp_view(
e, temporary_storage_provider
),
move_schema_columns_to_partitioned_by,
]
),
exp.DateFromParts: rename_func("MAKE_DATE"),
exp.DateTrunc: lambda self, e: self.func("TRUNC", e.this, e.args.get("unit")),
exp.DayOfMonth: rename_func("DAYOFMONTH"),
@ -251,12 +272,6 @@ class Spark2(Hive):
return self.func("STRUCT", *args)
def temporary_storage_provider(self, expression: exp.Create) -> exp.Create:
# spark2, spark, Databricks require a storage provider for temporary tables
provider = exp.FileFormatProperty(this=exp.Literal.string("parquet"))
expression.args["properties"].append("expressions", provider)
return expression
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
if is_parse_json(expression.this):
schema = f"'{self.sql(expression, 'to')}'"

1
sqlglot/dialects/sqlite.py
View file

@ -132,6 +132,7 @@ class SQLite(Dialect):
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
exp.DateAdd: _date_add_sql,
exp.DateStrToDate: lambda self, e: self.sql(e, "this"),
exp.If: rename_func("IIF"),
exp.ILike: no_ilike_sql,
exp.JSONExtract: _json_extract_sql,
exp.JSONExtractScalar: arrow_json_extract_sql,

6
sqlglot/dialects/tableau.py
View file

@ -1,10 +1,14 @@
from __future__ import annotations
from sqlglot import exp, generator, parser, transforms
from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import Dialect, rename_func
class Tableau(Dialect):
class Tokenizer(tokens.Tokenizer):
IDENTIFIERS = [("[", "]")]
QUOTES = ["'", '"']
class Generator(generator.Generator):
JOIN_HINTS = False
TABLE_HINTS = False

1
sqlglot/dialects/teradata.py
View file

@ -74,6 +74,7 @@ class Teradata(Dialect):
class Parser(parser.Parser):
TABLESAMPLE_CSV = True
VALUES_FOLLOWED_BY_PAREN = False
CHARSET_TRANSLATORS = {
"GRAPHIC_TO_KANJISJIS",

1
sqlglot/dialects/tsql.py
View file

@ -457,7 +457,6 @@ class TSQL(Dialect):
"FORMAT": _parse_format,
"GETDATE": exp.CurrentTimestamp.from_arg_list,
"HASHBYTES": _parse_hashbytes,
"IIF": exp.If.from_arg_list,
"ISNULL": exp.Coalesce.from_arg_list,
"JSON_QUERY": parser.parse_extract_json_with_path(exp.JSONExtract),
"JSON_VALUE": parser.parse_extract_json_with_path(exp.JSONExtractScalar),

70
sqlglot/expressions.py
View file

@ -1090,6 +1090,11 @@ class Create(DDL):
"clone": False,
}
@property
def kind(self) -> t.Optional[str]:
kind = self.args.get("kind")
return kind and kind.upper()
# https://docs.snowflake.com/en/sql-reference/sql/create-clone
# https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_table_clone_statement
@ -4626,6 +4631,11 @@ class CountIf(AggFunc):
_sql_names = ["COUNT_IF", "COUNTIF"]
# cube root
class Cbrt(Func):
pass
class CurrentDate(Func):
arg_types = {"this": False}
@ -4728,7 +4738,7 @@ class Extract(Func):
class Timestamp(Func):
arg_types = {"this": False, "expression": False}
arg_types = {"this": False, "expression": False, "with_tz": False}
class TimestampAdd(Func, TimeUnit):
@ -4833,7 +4843,7 @@ class Posexplode(Explode):
pass
class PosexplodeOuter(Posexplode):
class PosexplodeOuter(Posexplode, ExplodeOuter):
pass
@ -4868,6 +4878,7 @@ class Xor(Connector, Func):
class If(Func):
arg_types = {"this": True, "true": True, "false": False}
_sql_names = ["IF", "IIF"]
class Nullif(Func):
@ -6883,6 +6894,7 @@ def replace_tables(
table = to_table(
new_name,
**{k: v for k, v in node.args.items() if k not in TABLE_PARTS},
dialect=dialect,
)
table.add_comments([original])
return table
@ -7072,6 +7084,60 @@ def cast_unless(
return cast(expr, to, **opts)
def array(
*expressions: ExpOrStr, copy: bool = True, dialect: DialectType = None, **kwargs
) -> Array:
"""
Returns an array.
Examples:
>>> array(1, 'x').sql()
'ARRAY(1, x)'
Args:
expressions: the expressions to add to the array.
copy: whether or not to copy the argument expressions.
dialect: the source dialect.
kwargs: the kwargs used to instantiate the function of interest.
Returns:
An array expression.
"""
return Array(
expressions=[
maybe_parse(expression, copy=copy, dialect=dialect, **kwargs)
for expression in expressions
]
)
def tuple_(
*expressions: ExpOrStr, copy: bool = True, dialect: DialectType = None, **kwargs
) -> Tuple:
"""
Returns an tuple.
Examples:
>>> tuple_(1, 'x').sql()
'(1, x)'
Args:
expressions: the expressions to add to the tuple.
copy: whether or not to copy the argument expressions.
dialect: the source dialect.
kwargs: the kwargs used to instantiate the function of interest.
Returns:
A tuple expression.
"""
return Tuple(
expressions=[
maybe_parse(expression, copy=copy, dialect=dialect, **kwargs)
for expression in expressions
]
)
def true() -> Boolean:
"""
Returns a true Boolean expression.

3
sqlglot/generator.py
View file

@ -124,6 +124,7 @@ class Generator(metaclass=_Generator):
exp.StabilityProperty: lambda self, e: e.name,
exp.TemporaryProperty: lambda self, e: "TEMPORARY",
exp.TitleColumnConstraint: lambda self, e: f"TITLE {self.sql(e, 'this')}",
exp.Timestamp: lambda self, e: self.func("TIMESTAMP", e.this, e.expression),
exp.ToTableProperty: lambda self, e: f"TO {self.sql(e.this)}",
exp.TransformModelProperty: lambda self, e: self.func("TRANSFORM", *e.expressions),
exp.TransientProperty: lambda self, e: "TRANSIENT",
@ -3360,7 +3361,7 @@ class Generator(metaclass=_Generator):
return self.sql(arg)
cond_for_null = arg.is_(exp.null())
return self.sql(exp.func("IF", cond_for_null, exp.null(), exp.Array(expressions=[arg])))
return self.sql(exp.func("IF", cond_for_null, exp.null(), exp.array(arg, copy=False)))
def tsordstotime_sql(self, expression: exp.TsOrDsToTime) -> str:
this = expression.this

30
sqlglot/helper.py
View file

@ -6,7 +6,7 @@ import logging
import re
import sys
import typing as t
from collections.abc import Collection
from collections.abc import Collection, Set
from contextlib import contextmanager
from copy import copy
from enum import Enum
@ -496,3 +496,31 @@ DATE_UNITS = {"day", "week", "month", "quarter", "year", "year_month"}
def is_date_unit(expression: t.Optional[exp.Expression]) -> bool:
return expression is not None and expression.name.lower() in DATE_UNITS
K = t.TypeVar("K")
V = t.TypeVar("V")
class SingleValuedMapping(t.Mapping[K, V]):
"""
Mapping where all keys return the same value.
This rigamarole is meant to avoid copying keys, which was originally intended
as an optimization while qualifying columns for tables with lots of columns.
"""
def __init__(self, keys: t.Collection[K], value: V):
self._keys = keys if isinstance(keys, Set) else set(keys)
self._value = value
def __getitem__(self, key: K) -> V:
if key in self._keys:
return self._value
raise KeyError(key)
def __len__(self) -> int:
return len(self._keys)
def __iter__(self) -> t.Iterator[K]:
return iter(self._keys)

8
sqlglot/lineage.py
View file

@ -153,7 +153,7 @@ def lineage(
raise ValueError(f"Could not find {column} in {scope.expression}")
for s in scope.union_scopes:
to_node(index, scope=s, upstream=upstream)
to_node(index, scope=s, upstream=upstream, alias=alias)
return upstream
@ -209,7 +209,11 @@ def lineage(
if isinstance(source, Scope):
# The table itself came from a more specific scope. Recurse into that one using the unaliased column name.
to_node(
c.name, scope=source, scope_name=table, upstream=node, alias=aliases.get(table)
c.name,
scope=source,
scope_name=table,
upstream=node,
alias=aliases.get(table) or alias,
)
else:
# The source is not a scope - we've reached the end of the line. At this point, if a source is not found

5
sqlglot/optimizer/annotate_types.py
View file

@ -204,7 +204,6 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.TimeAdd,
exp.TimeStrToTime,
exp.TimeSub,
exp.Timestamp,
exp.TimestampAdd,
exp.TimestampSub,
exp.UnixToTime,
@ -276,6 +275,10 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
exp.Nullif: lambda self, e: self._annotate_by_args(e, "this", "expression"),
exp.Slice: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN),
exp.Sum: lambda self, e: self._annotate_by_args(e, "this", "expressions", promote=True),
exp.Timestamp: lambda self, e: self._annotate_with_type(
e,
exp.DataType.Type.TIMESTAMPTZ if e.args.get("with_tz") else exp.DataType.Type.TIMESTAMP,
),
exp.TryCast: lambda self, e: self._annotate_with_type(e, e.args["to"]),
exp.VarMap: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.MAP),
exp.Struct: lambda self, e: self._annotate_by_args(e, "expressions", struct=True),

10
sqlglot/optimizer/canonicalize.py
View file

@ -38,7 +38,12 @@ def replace_date_funcs(node: exp.Expression) -> exp.Expression:
if isinstance(node, exp.Date) and not node.expressions and not node.args.get("zone"):
return exp.cast(node.this, to=exp.DataType.Type.DATE)
if isinstance(node, exp.Timestamp) and not node.expression:
return exp.cast(node.this, to=exp.DataType.Type.TIMESTAMP)
if not node.type:
from sqlglot.optimizer.annotate_types import annotate_types
node = annotate_types(node)
return exp.cast(node.this, to=node.type or exp.DataType.Type.TIMESTAMP)
return node
@ -76,9 +81,8 @@ def coerce_type(node: exp.Expression) -> exp.Expression:
def remove_redundant_casts(expression: exp.Expression) -> exp.Expression:
if (
isinstance(expression, exp.Cast)
and expression.to.type
and expression.this.type
and expression.to.type.this == expression.this.type.this
and expression.to.this == expression.this.type.this
):
return expression.this
return expression

50
sqlglot/optimizer/qualify_columns.py
View file

@ -6,7 +6,7 @@ import typing as t
from sqlglot import alias, exp
from sqlglot.dialects.dialect import Dialect, DialectType
from sqlglot.errors import OptimizeError
from sqlglot.helper import seq_get
from sqlglot.helper import seq_get, SingleValuedMapping
from sqlglot.optimizer.scope import Scope, build_scope, traverse_scope, walk_in_scope
from sqlglot.optimizer.simplify import simplify_parens
from sqlglot.schema import Schema, ensure_schema
@ -586,8 +586,8 @@ class Resolver:
def __init__(self, scope: Scope, schema: Schema, infer_schema: bool = True):
self.scope = scope
self.schema = schema
self._source_columns: t.Optional[t.Dict[str, t.List[str]]] = None
self._unambiguous_columns: t.Optional[t.Dict[str, str]] = None
self._source_columns: t.Optional[t.Dict[str, t.Sequence[str]]] = None
self._unambiguous_columns: t.Optional[t.Mapping[str, str]] = None
self._all_columns: t.Optional[t.Set[str]] = None
self._infer_schema = infer_schema
@ -640,7 +640,7 @@ class Resolver:
}
return self._all_columns
def get_source_columns(self, name: str, only_visible: bool = False) -> t.List[str]:
def get_source_columns(self, name: str, only_visible: bool = False) -> t.Sequence[str]:
"""Resolve the source columns for a given source `name`."""
if name not in self.scope.sources:
raise OptimizeError(f"Unknown table: {name}")
@ -662,10 +662,15 @@ class Resolver:
else:
column_aliases = []
# If the source's columns are aliased, their aliases shadow the corresponding column names
return [alias or name for (name, alias) in itertools.zip_longest(columns, column_aliases)]
if column_aliases:
# If the source's columns are aliased, their aliases shadow the corresponding column names.
# This can be expensive if there are lots of columns, so only do this if column_aliases exist.
return [
alias or name for (name, alias) in itertools.zip_longest(columns, column_aliases)
]
return columns
def _get_all_source_columns(self) -> t.Dict[str, t.List[str]]:
def _get_all_source_columns(self) -> t.Dict[str, t.Sequence[str]]:
if self._source_columns is None:
self._source_columns = {
source_name: self.get_source_columns(source_name)
@ -676,8 +681,8 @@ class Resolver:
return self._source_columns
def _get_unambiguous_columns(
self, source_columns: t.Dict[str, t.List[str]]
) -> t.Dict[str, str]:
self, source_columns: t.Dict[str, t.Sequence[str]]
) -> t.Mapping[str, str]:
"""
Find all the unambiguous columns in sources.
@ -693,12 +698,17 @@ class Resolver:
source_columns_pairs = list(source_columns.items())
first_table, first_columns = source_columns_pairs[0]
unambiguous_columns = {col: first_table for col in self._find_unique_columns(first_columns)}
if len(source_columns_pairs) == 1:
# Performance optimization - avoid copying first_columns if there is only one table.
return SingleValuedMapping(first_columns, first_table)
unambiguous_columns = {col: first_table for col in first_columns}
all_columns = set(unambiguous_columns)
for table, columns in source_columns_pairs[1:]:
unique = self._find_unique_columns(columns)
ambiguous = set(all_columns).intersection(unique)
unique = set(columns)
ambiguous = all_columns.intersection(unique)
all_columns.update(columns)
for column in ambiguous:
@ -707,19 +717,3 @@ class Resolver:
unambiguous_columns[column] = table
return unambiguous_columns
@staticmethod
def _find_unique_columns(columns: t.Collection[str]) -> t.Set[str]:
"""
Find the unique columns in a list of columns.
Example:
>>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))
['a', 'c']
This is necessary because duplicate column names are ambiguous.
"""
counts: t.Dict[str, int] = {}
for column in columns:
counts[column] = counts.get(column, 0) + 1
return {column for column, count in counts.items() if count == 1}

52
sqlglot/parser.py
View file

@ -29,8 +29,8 @@ def parse_var_map(args: t.List) -> exp.StarMap | exp.VarMap:
values.append(args[i + 1])
return exp.VarMap(
keys=exp.Array(expressions=keys),
values=exp.Array(expressions=values),
keys=exp.array(*keys, copy=False),
values=exp.array(*values, copy=False),
)
@ -638,6 +638,8 @@ class Parser(metaclass=_Parser):
TokenType.NOT: lambda self: self.expression(exp.Not, this=self._parse_equality()),
TokenType.TILDA: lambda self: self.expression(exp.BitwiseNot, this=self._parse_unary()),
TokenType.DASH: lambda self: self.expression(exp.Neg, this=self._parse_unary()),
TokenType.PIPE_SLASH: lambda self: self.expression(exp.Sqrt, this=self._parse_unary()),
TokenType.DPIPE_SLASH: lambda self: self.expression(exp.Cbrt, this=self._parse_unary()),
}
PRIMARY_PARSERS = {
@ -1000,9 +1002,13 @@ class Parser(metaclass=_Parser):
MODIFIERS_ATTACHED_TO_UNION = True
UNION_MODIFIERS = {"order", "limit", "offset"}
# parses no parenthesis if statements as commands
# Parses no parenthesis if statements as commands
NO_PAREN_IF_COMMANDS = True
# Whether or not a VALUES keyword needs to be followed by '(' to form a VALUES clause.
# If this is True and '(' is not found, the keyword will be treated as an identifier
VALUES_FOLLOWED_BY_PAREN = True
__slots__ = (
"error_level",
"error_message_context",
@ -2058,7 +2064,7 @@ class Parser(metaclass=_Parser):
partition=self._parse_partition(),
where=self._match_pair(TokenType.REPLACE, TokenType.WHERE)
and self._parse_conjunction(),
expression=self._parse_ddl_select(),
expression=self._parse_derived_table_values() or self._parse_ddl_select(),
conflict=self._parse_on_conflict(),
returning=returning or self._parse_returning(),
overwrite=overwrite,
@ -2267,8 +2273,7 @@ class Parser(metaclass=_Parser):
self._match_r_paren()
return self.expression(exp.Tuple, expressions=expressions)
# In presto we can have VALUES 1, 2 which results in 1 column & 2 rows.
# https://prestodb.io/docs/current/sql/values.html
# In some dialects we can have VALUES 1, 2 which results in 1 column & 2 rows.
return self.expression(exp.Tuple, expressions=[self._parse_expression()])
def _parse_projections(self) -> t.List[exp.Expression]:
@ -2367,12 +2372,8 @@ class Parser(metaclass=_Parser):
# We return early here so that the UNION isn't attached to the subquery by the
# following call to _parse_set_operations, but instead becomes the parent node
return self._parse_subquery(this, parse_alias=parse_subquery_alias)
elif self._match(TokenType.VALUES):
this = self.expression(
exp.Values,
expressions=self._parse_csv(self._parse_value),
alias=self._parse_table_alias(),
)
elif self._match(TokenType.VALUES, advance=False):
this = self._parse_derived_table_values()
elif from_:
this = exp.select("*").from_(from_.this, copy=False)
else:
@ -2969,7 +2970,7 @@ class Parser(metaclass=_Parser):
def _parse_derived_table_values(self) -> t.Optional[exp.Values]:
is_derived = self._match_pair(TokenType.L_PAREN, TokenType.VALUES)
if not is_derived and not self._match(TokenType.VALUES):
if not is_derived and not self._match_text_seq("VALUES"):
return None
expressions = self._parse_csv(self._parse_value)
@ -3655,8 +3656,15 @@ class Parser(metaclass=_Parser):
def _parse_type(self, parse_interval: bool = True) -> t.Optional[exp.Expression]:
interval = parse_interval and self._parse_interval()
if interval:
# Convert INTERVAL 'val_1' unit_1 ... 'val_n' unit_n into a sum of intervals
while self._match_set((TokenType.STRING, TokenType.NUMBER), advance=False):
# Convert INTERVAL 'val_1' unit_1 [+] ... [+] 'val_n' unit_n into a sum of intervals
while True:
index = self._index
self._match(TokenType.PLUS)
if not self._match_set((TokenType.STRING, TokenType.NUMBER), advance=False):
self._retreat(index)
break
interval = self.expression( # type: ignore
exp.Add, this=interval, expression=self._parse_interval(match_interval=False)
)
@ -3872,9 +3880,15 @@ class Parser(metaclass=_Parser):
def _parse_column_reference(self) -> t.Optional[exp.Expression]:
this = self._parse_field()
if isinstance(this, exp.Identifier):
this = self.expression(exp.Column, this=this)
return this
if (
not this
and self._match(TokenType.VALUES, advance=False)
and self.VALUES_FOLLOWED_BY_PAREN
and (not self._next or self._next.token_type != TokenType.L_PAREN)
):
this = self._parse_id_var()
return self.expression(exp.Column, this=this) if isinstance(this, exp.Identifier) else this
def _parse_column_ops(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
this = self._parse_bracket(this)
@ -5511,7 +5525,7 @@ class Parser(metaclass=_Parser):
then = self.expression(
exp.Insert,
this=self._parse_value(),
expression=self._match(TokenType.VALUES) and self._parse_value(),
expression=self._match_text_seq("VALUES") and self._parse_value(),
)
elif self._match(TokenType.UPDATE):
expressions = self._parse_star()

4
sqlglot/schema.py
View file

@ -49,7 +49,7 @@ class Schema(abc.ABC):
only_visible: bool = False,
dialect: DialectType = None,
normalize: t.Optional[bool] = None,
) -> t.List[str]:
) -> t.Sequence[str]:
"""
Get the column names for a table.
@ -60,7 +60,7 @@ class Schema(abc.ABC):
normalize: whether to normalize identifiers according to the dialect of interest.
Returns:
The list of column names.
The sequence of column names.
"""
@abc.abstractmethod

2
sqlglot/tokens.py
View file

@ -57,6 +57,8 @@ class TokenType(AutoName):
AMP = auto()
DPIPE = auto()
PIPE = auto()
PIPE_SLASH = auto()
DPIPE_SLASH = auto()
CARET = auto()
TILDA = auto()
ARROW = auto()

94
sqlglot/transforms.py
View file

@ -213,6 +213,19 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
is_posexplode = isinstance(explode, exp.Posexplode)
explode_arg = explode.this
if isinstance(explode, exp.ExplodeOuter):
bracket = explode_arg[0]
bracket.set("safe", True)
bracket.set("offset", True)
explode_arg = exp.func(
"IF",
exp.func(
"ARRAY_SIZE", exp.func("COALESCE", explode_arg, exp.Array())
).eq(0),
exp.array(bracket, copy=False),
explode_arg,
)
# This ensures that we won't use [POS]EXPLODE's argument as a new selection
if isinstance(explode_arg, exp.Column):
taken_select_names.add(explode_arg.output_name)
@ -466,6 +479,87 @@ def unqualify_columns(expression: exp.Expression) -> exp.Expression:
return expression
def remove_unique_constraints(expression: exp.Expression) -> exp.Expression:
assert isinstance(expression, exp.Create)
for constraint in expression.find_all(exp.UniqueColumnConstraint):
if constraint.parent:
constraint.parent.pop()
return expression
def ctas_with_tmp_tables_to_create_tmp_view(
expression: exp.Expression,
tmp_storage_provider: t.Callable[[exp.Expression], exp.Expression] = lambda e: e,
) -> exp.Expression:
assert isinstance(expression, exp.Create)
properties = expression.args.get("properties")
temporary = any(
isinstance(prop, exp.TemporaryProperty)
for prop in (properties.expressions if properties else [])
)
# CTAS with temp tables map to CREATE TEMPORARY VIEW
if expression.kind == "TABLE" and temporary:
if expression.expression:
return exp.Create(
kind="TEMPORARY VIEW",
this=expression.this,
expression=expression.expression,
)
return tmp_storage_provider(expression)
return expression
def move_schema_columns_to_partitioned_by(expression: exp.Expression) -> exp.Expression:
"""
In Hive, the PARTITIONED BY property acts as an extension of a table's schema. When the
PARTITIONED BY value is an array of column names, they are transformed into a schema.
The corresponding columns are removed from the create statement.
"""
assert isinstance(expression, exp.Create)
has_schema = isinstance(expression.this, exp.Schema)
is_partitionable = expression.kind in {"TABLE", "VIEW"}
if has_schema and is_partitionable:
prop = expression.find(exp.PartitionedByProperty)
if prop and prop.this and not isinstance(prop.this, exp.Schema):
schema = expression.this
columns = {v.name.upper() for v in prop.this.expressions}
partitions = [col for col in schema.expressions if col.name.upper() in columns]
schema.set("expressions", [e for e in schema.expressions if e not in partitions])
prop.replace(exp.PartitionedByProperty(this=exp.Schema(expressions=partitions)))
expression.set("this", schema)
return expression
def move_partitioned_by_to_schema_columns(expression: exp.Expression) -> exp.Expression:
"""
Spark 3 supports both "HIVEFORMAT" and "DATASOURCE" formats for CREATE TABLE.
Currently, SQLGlot uses the DATASOURCE format for Spark 3.
"""
assert isinstance(expression, exp.Create)
prop = expression.find(exp.PartitionedByProperty)
if (
prop
and prop.this
and isinstance(prop.this, exp.Schema)
and all(isinstance(e, exp.ColumnDef) and e.args.get("kind") for e in prop.this.expressions)
):
prop_this = exp.Tuple(
expressions=[exp.to_identifier(e.this) for e in prop.this.expressions]
)
schema = expression.this
for e in prop.this.expressions:
schema.append("expressions", e)
prop.set("this", prop_this)
return expression
def preprocess(
transforms: t.List[t.Callable[[exp.Expression], exp.Expression]],
) -> t.Callable[[Generator, exp.Expression], str]:

274
tests/dialects/test_bigquery.py
View file

@ -18,78 +18,6 @@ class TestBigQuery(Validator):
maxDiff = None
def test_bigquery(self):
self.validate_identity("ARRAY_AGG(x IGNORE NULLS LIMIT 1)")
self.validate_identity("ARRAY_AGG(x IGNORE NULLS ORDER BY x LIMIT 1)")
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY x LIMIT 1)")
self.validate_identity("ARRAY_AGG(x IGNORE NULLS)")
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS HAVING MAX x ORDER BY x LIMIT 1)")
self.validate_all(
"SELECT SUM(x IGNORE NULLS) AS x",
read={
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
},
write={
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
},
)
self.validate_all(
"SELECT SUM(x RESPECT NULLS) AS x",
read={
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
},
write={
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
},
)
self.validate_all(
"SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
write={
"bigquery": "SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
"duckdb": "SELECT QUANTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
"spark": "SELECT PERCENTILE_CONT(x, 0.5) RESPECT NULLS OVER ()",
},
)
self.validate_all(
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
write={
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 10) AS x",
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 10) IGNORE NULLS AS x",
},
)
self.validate_all(
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
write={
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 1, 10) AS x",
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 1, 10) IGNORE NULLS AS x",
},
)
self.validate_identity("SELECT COUNT(x RESPECT NULLS)")
self.validate_identity("SELECT LAST_VALUE(x IGNORE NULLS) OVER y AS x")
self.validate_identity(
"create or replace view test (tenant_id OPTIONS(description='Test description on table creation')) select 1 as tenant_id, 1 as customer_id;",
"CREATE OR REPLACE VIEW test (tenant_id OPTIONS (description='Test description on table creation')) AS SELECT 1 AS tenant_id, 1 AS customer_id",
)
with self.assertLogs(helper_logger) as cm:
statements = parse(
"""
@ -131,19 +59,12 @@ class TestBigQuery(Validator):
self.validate_all(
"a[0]",
read={
"bigquery": "a[0]",
"duckdb": "a[1]",
"presto": "a[1]",
},
)
self.validate_identity(
"select array_contains([1, 2, 3], 1)",
"SELECT EXISTS(SELECT 1 FROM UNNEST([1, 2, 3]) AS _col WHERE _col = 1)",
)
self.validate_identity("CREATE SCHEMA x DEFAULT COLLATE 'en'")
self.validate_identity("CREATE TABLE x (y INT64) DEFAULT COLLATE 'en'")
self.validate_identity("PARSE_JSON('{}', wide_number_mode => 'exact')")
with self.assertRaises(TokenError):
transpile("'\\'", read="bigquery")
@ -179,6 +100,16 @@ class TestBigQuery(Validator):
)
assert "'END FOR'" in cm.output[0]
self.validate_identity("CREATE SCHEMA x DEFAULT COLLATE 'en'")
self.validate_identity("CREATE TABLE x (y INT64) DEFAULT COLLATE 'en'")
self.validate_identity("PARSE_JSON('{}', wide_number_mode => 'exact')")
self.validate_identity("FOO(values)")
self.validate_identity("STRUCT(values AS value)")
self.validate_identity("ARRAY_AGG(x IGNORE NULLS LIMIT 1)")
self.validate_identity("ARRAY_AGG(x IGNORE NULLS ORDER BY x LIMIT 1)")
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY x LIMIT 1)")
self.validate_identity("ARRAY_AGG(x IGNORE NULLS)")
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS HAVING MAX x ORDER BY x LIMIT 1)")
self.validate_identity("SELECT * FROM dataset.my_table TABLESAMPLE SYSTEM (10 PERCENT)")
self.validate_identity("TIME('2008-12-25 15:30:00+08')")
self.validate_identity("TIME('2008-12-25 15:30:00+08', 'America/Los_Angeles')")
@ -237,6 +168,13 @@ class TestBigQuery(Validator):
self.validate_identity("SELECT TIMESTAMP_SECONDS(2) AS t")
self.validate_identity("SELECT TIMESTAMP_MILLIS(2) AS t")
self.validate_identity("""SELECT JSON_EXTRACT_SCALAR('{"a": 5}', '$.a')""")
self.validate_identity("UPDATE x SET y = NULL")
self.validate_identity("LOG(n, b)")
self.validate_identity("SELECT COUNT(x RESPECT NULLS)")
self.validate_identity("SELECT LAST_VALUE(x IGNORE NULLS) OVER y AS x")
self.validate_identity(
"SELECT * FROM test QUALIFY a IS DISTINCT FROM b WINDOW c AS (PARTITION BY d)"
)
self.validate_identity(
"FOR record IN (SELECT word, word_count FROM bigquery-public-data.samples.shakespeare LIMIT 5) DO SELECT record.word, record.word_count"
)
@ -264,6 +202,14 @@ class TestBigQuery(Validator):
self.validate_identity(
"""SELECT JSON_EXTRACT_SCALAR('5')""", """SELECT JSON_EXTRACT_SCALAR('5', '$')"""
)
self.validate_identity(
"select array_contains([1, 2, 3], 1)",
"SELECT EXISTS(SELECT 1 FROM UNNEST([1, 2, 3]) AS _col WHERE _col = 1)",
)
self.validate_identity(
"create or replace view test (tenant_id OPTIONS(description='Test description on table creation')) select 1 as tenant_id, 1 as customer_id;",
"CREATE OR REPLACE VIEW test (tenant_id OPTIONS (description='Test description on table creation')) AS SELECT 1 AS tenant_id, 1 AS customer_id",
)
self.validate_identity(
"SELECT SPLIT(foo)",
"SELECT SPLIT(foo, ',')",
@ -312,7 +258,81 @@ class TestBigQuery(Validator):
"SELECT * FROM UNNEST(x) WITH OFFSET EXCEPT DISTINCT SELECT * FROM UNNEST(y) WITH OFFSET",
"SELECT * FROM UNNEST(x) WITH OFFSET AS offset EXCEPT DISTINCT SELECT * FROM UNNEST(y) WITH OFFSET AS offset",
)
self.validate_identity(
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) d, COUNT(*) e FOR c IN ('x', 'y'))",
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) AS d, COUNT(*) AS e FOR c IN ('x', 'y'))",
)
self.validate_identity(
r"REGEXP_EXTRACT(svc_plugin_output, r'\\\((.*)')",
r"REGEXP_EXTRACT(svc_plugin_output, '\\\\\\((.*)')",
)
self.validate_all(
"TIMESTAMP(x)",
write={
"bigquery": "TIMESTAMP(x)",
"duckdb": "CAST(x AS TIMESTAMPTZ)",
"presto": "CAST(x AS TIMESTAMP WITH TIME ZONE)",
},
)
self.validate_all(
"SELECT SUM(x IGNORE NULLS) AS x",
read={
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
},
write={
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
},
)
self.validate_all(
"SELECT SUM(x RESPECT NULLS) AS x",
read={
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
},
write={
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
},
)
self.validate_all(
"SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
write={
"bigquery": "SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
"duckdb": "SELECT QUANTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
"spark": "SELECT PERCENTILE_CONT(x, 0.5) RESPECT NULLS OVER ()",
},
)
self.validate_all(
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
write={
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 10) AS x",
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 10) IGNORE NULLS AS x",
},
)
self.validate_all(
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
write={
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 1, 10) AS x",
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 1, 10) IGNORE NULLS AS x",
},
)
self.validate_all(
"SELECT * FROM Produce UNPIVOT((first_half_sales, second_half_sales) FOR semesters IN ((Q1, Q2) AS 'semester_1', (Q3, Q4) AS 'semester_2'))",
read={
@ -464,7 +484,6 @@ class TestBigQuery(Validator):
"duckdb": "SELECT * FROM t WHERE EXISTS(SELECT * FROM UNNEST(nums) AS _t(x) WHERE x > 1)",
},
)
self.validate_identity("UPDATE x SET y = NULL")
self.validate_all(
"NULL",
read={
@ -620,6 +639,14 @@ class TestBigQuery(Validator):
"spark": "WITH cte AS (SELECT ARRAY(1, 2, 3) AS arr) SELECT EXPLODE(arr) FROM cte"
},
)
self.validate_all(
"SELECT IF(pos = pos_2, col, NULL) AS col FROM UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], []))) - 1)) AS pos CROSS JOIN UNNEST(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) AS col WITH OFFSET AS pos_2 WHERE pos = pos_2 OR (pos > (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1) AND pos_2 = (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1))",
read={"spark": "select explode_outer([])"},
)
self.validate_all(
"SELECT IF(pos = pos_2, col, NULL) AS col, IF(pos = pos_2, pos_2, NULL) AS pos_2 FROM UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], []))) - 1)) AS pos CROSS JOIN UNNEST(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) AS col WITH OFFSET AS pos_2 WHERE pos = pos_2 OR (pos > (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1) AND pos_2 = (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1))",
read={"spark": "select posexplode_outer([])"},
)
self.validate_all(
"SELECT AS STRUCT ARRAY(SELECT AS STRUCT b FROM x) AS y FROM z",
write={
@ -660,10 +687,6 @@ class TestBigQuery(Validator):
"bigquery": "SELECT ARRAY(SELECT AS STRUCT 1 AS a, 2 AS b)",
},
)
self.validate_identity(
r"REGEXP_EXTRACT(svc_plugin_output, r'\\\((.*)')",
r"REGEXP_EXTRACT(svc_plugin_output, '\\\\\\((.*)')",
)
self.validate_all(
"REGEXP_CONTAINS('foo', '.*')",
read={
@ -986,9 +1009,6 @@ class TestBigQuery(Validator):
"postgres": "CURRENT_DATE AT TIME ZONE 'UTC'",
},
)
self.validate_identity(
"SELECT * FROM test QUALIFY a IS DISTINCT FROM b WINDOW c AS (PARTITION BY d)"
)
self.validate_all(
"SELECT a FROM test WHERE a = 1 GROUP BY a HAVING a = 2 QUALIFY z ORDER BY a LIMIT 10",
write={
@ -997,45 +1017,20 @@ class TestBigQuery(Validator):
},
)
self.validate_all(
"SELECT cola, colb FROM (VALUES (1, 'test')) AS tab(cola, colb)",
write={
"spark": "SELECT cola, colb FROM VALUES (1, 'test') AS tab(cola, colb)",
"SELECT cola, colb FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb)])",
read={
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb)])",
"snowflake": "SELECT cola, colb FROM (VALUES (1, 'test')) AS tab(cola, colb)",
},
)
self.validate_all(
"SELECT cola, colb FROM (VALUES (1, 'test')) AS tab",
write={
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS _c0, 'test' AS _c1)])",
},
)
self.validate_all(
"SELECT cola, colb FROM (VALUES (1, 'test'))",
write={
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS _c0, 'test' AS _c1)])",
"spark": "SELECT cola, colb FROM VALUES (1, 'test') AS tab(cola, colb)",
},
)
self.validate_all(
"SELECT * FROM UNNEST([STRUCT(1 AS id)]) CROSS JOIN UNNEST([STRUCT(1 AS id)])",
read={
"bigquery": "SELECT * FROM UNNEST([STRUCT(1 AS id)]) CROSS JOIN UNNEST([STRUCT(1 AS id)])",
"postgres": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
},
)
self.validate_all(
"SELECT cola, colb, colc FROM (VALUES (1, 'test', NULL)) AS tab(cola, colb, colc)",
write={
"spark": "SELECT cola, colb, colc FROM VALUES (1, 'test', NULL) AS tab(cola, colb, colc)",
"bigquery": "SELECT cola, colb, colc FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb, NULL AS colc)])",
"snowflake": "SELECT cola, colb, colc FROM (VALUES (1, 'test', NULL)) AS tab(cola, colb, colc)",
},
)
self.validate_all(
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) d, COUNT(*) e FOR c IN ('x', 'y'))",
write={
"bigquery": "SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) AS d, COUNT(*) AS e FOR c IN ('x', 'y'))",
},
)
self.validate_all(
"SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
write={
@ -1091,8 +1086,6 @@ WHERE
pretty=True,
)
self.validate_identity("LOG(n, b)")
def test_user_defined_functions(self):
self.validate_identity(
"CREATE TEMPORARY FUNCTION a(x FLOAT64, y FLOAT64) RETURNS FLOAT64 NOT DETERMINISTIC LANGUAGE js AS 'return x*y;'"
@ -1114,35 +1107,22 @@ WHERE
)
def test_remove_precision_parameterized_types(self):
self.validate_all(
"SELECT CAST(1 AS NUMERIC(10, 2))",
write={
"bigquery": "SELECT CAST(1 AS NUMERIC)",
},
)
self.validate_all(
"CREATE TABLE test (a NUMERIC(10, 2))",
write={
"bigquery": "CREATE TABLE test (a NUMERIC(10, 2))",
},
)
self.validate_all(
"SELECT CAST('1' AS STRING(10)) UNION ALL SELECT CAST('2' AS STRING(10))",
write={
"bigquery": "SELECT CAST('1' AS STRING) UNION ALL SELECT CAST('2' AS STRING)",
},
)
self.validate_all(
"SELECT cola FROM (SELECT CAST('1' AS STRING(10)) AS cola UNION ALL SELECT CAST('2' AS STRING(10)) AS cola)",
write={
"bigquery": "SELECT cola FROM (SELECT CAST('1' AS STRING) AS cola UNION ALL SELECT CAST('2' AS STRING) AS cola)",
},
)
self.validate_all(
self.validate_identity("CREATE TABLE test (a NUMERIC(10, 2))")
self.validate_identity(
"INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING(10)), CAST(14 AS STRING(10)))",
write={
"bigquery": "INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING), CAST(14 AS STRING))",
},
"INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING), CAST(14 AS STRING))",
)
self.validate_identity(
"SELECT CAST(1 AS NUMERIC(10, 2))",
"SELECT CAST(1 AS NUMERIC)",
)
self.validate_identity(
"SELECT CAST('1' AS STRING(10)) UNION ALL SELECT CAST('2' AS STRING(10))",
"SELECT CAST('1' AS STRING) UNION ALL SELECT CAST('2' AS STRING)",
)
self.validate_identity(
"SELECT cola FROM (SELECT CAST('1' AS STRING(10)) AS cola UNION ALL SELECT CAST('2' AS STRING(10)) AS cola)",
"SELECT cola FROM (SELECT CAST('1' AS STRING) AS cola UNION ALL SELECT CAST('2' AS STRING) AS cola)",
)
def test_models(self):

1
tests/dialects/test_clickhouse.py
View file

@ -369,6 +369,7 @@ class TestClickhouse(Validator):
"SELECT STARTS_WITH('a', 'b'), STARTSWITH('a', 'b')",
write={"clickhouse": "SELECT startsWith('a', 'b'), startsWith('a', 'b')"},
)
self.validate_identity("SYSTEM STOP MERGES foo.bar", check_command_warning=True)
def test_cte(self):
self.validate_identity("WITH 'x' AS foo SELECT foo")

4
tests/dialects/test_dialect.py
View file

@ -2101,7 +2101,7 @@ SELECT
"databricks": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
"presto": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
"snowflake": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
"sqlite": "SELECT SUM(CASE WHEN col % 2 = 0 THEN 1 ELSE 0 END) FROM foo",
"sqlite": "SELECT SUM(IIF(col % 2 = 0, 1, 0)) FROM foo",
"tsql": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
},
)
@ -2116,7 +2116,7 @@ SELECT
"": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
"databricks": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
"presto": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
"sqlite": "SELECT SUM(CASE WHEN col % 2 = 0 THEN 1 ELSE 0 END) FILTER(WHERE col < 1000) FROM foo",
"sqlite": "SELECT SUM(IIF(col % 2 = 0, 1, 0)) FILTER(WHERE col < 1000) FROM foo",
"tsql": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
},
)

2
tests/dialects/test_hive.py
View file

@ -152,7 +152,7 @@ class TestHive(Validator):
"duckdb": "CREATE TABLE x (w TEXT)", # Partition columns should exist in table
"presto": "CREATE TABLE x (w VARCHAR, y INTEGER, z INTEGER) WITH (PARTITIONED_BY=ARRAY['y', 'z'])",
"hive": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
"spark": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
"spark": "CREATE TABLE x (w STRING, y INT, z INT) PARTITIONED BY (y, z)",
},
)
self.validate_all(

2
tests/dialects/test_postgres.py
View file

@ -8,6 +8,8 @@ class TestPostgres(Validator):
dialect = "postgres"
def test_postgres(self):
self.validate_identity("|/ x", "SQRT(x)")
self.validate_identity("||/ x", "CBRT(x)")
expr = parse_one(
"SELECT * FROM r CROSS JOIN LATERAL UNNEST(ARRAY[1]) AS s(location)", read="postgres"
)

2
tests/dialects/test_presto.py
View file

@ -424,7 +424,7 @@ class TestPresto(Validator):
"duckdb": "CREATE TABLE x (w TEXT, y INT, z INT)",
"presto": "CREATE TABLE x (w VARCHAR, y INTEGER, z INTEGER) WITH (PARTITIONED_BY=ARRAY['y', 'z'])",
"hive": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
"spark": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
"spark": "CREATE TABLE x (w STRING, y INT, z INT) PARTITIONED BY (y, z)",
},
)
self.validate_all(

96
tests/dialects/test_redshift.py
View file

@ -1,4 +1,4 @@
from sqlglot import transpile
from sqlglot import exp, parse_one, transpile
from tests.dialects.test_dialect import Validator
@ -381,8 +381,6 @@ class TestRedshift(Validator):
"SELECT DATEADD(DAY, 1, DATE('2023-01-01'))",
)
self.validate_identity("SELECT * FROM x AS a, a.b AS c, c.d.e AS f, f.g.h.i.j.k AS l")
self.validate_identity(
"""SELECT
c_name,
@ -408,8 +406,9 @@ ORDER BY
union_query = f"SELECT * FROM ({' UNION ALL '.join('SELECT ' + v for v in values)})"
self.assertEqual(transpile(values_query, write="redshift")[0], union_query)
self.validate_identity(
"SELECT * FROM (VALUES (1), (2))",
values_sql = transpile("SELECT * FROM (VALUES (1), (2))", write="redshift", pretty=True)[0]
self.assertEqual(
values_sql,
"""SELECT
*
FROM (
@ -419,69 +418,51 @@ FROM (
SELECT
2
)""",
pretty=True,
)
self.validate_identity("INSERT INTO t (a) VALUES (1), (2), (3)")
self.validate_identity("INSERT INTO t (a, b) VALUES (1, 2), (3, 4)")
self.validate_all(
"SELECT * FROM (VALUES (1, 2)) AS t",
"SELECT * FROM (SELECT 1, 2) AS t",
read={
"": "SELECT * FROM (VALUES (1, 2)) AS t",
},
write={
"redshift": "SELECT * FROM (SELECT 1, 2) AS t",
"mysql": "SELECT * FROM (SELECT 1, 2) AS t",
"presto": "SELECT * FROM (VALUES (1, 2)) AS t",
"presto": "SELECT * FROM (SELECT 1, 2) AS t",
},
)
self.validate_all(
"SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
write={
"redshift": "SELECT * FROM (SELECT 1 AS id) AS t1 CROSS JOIN (SELECT 1 AS id) AS t2",
"SELECT * FROM (SELECT 1 AS id) AS t1 CROSS JOIN (SELECT 1 AS id) AS t2",
read={
"": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
},
)
self.validate_all(
"SELECT a, b FROM (VALUES (1, 2)) AS t (a, b)",
write={
"redshift": "SELECT a, b FROM (SELECT 1 AS a, 2 AS b) AS t",
"SELECT a, b FROM (SELECT 1 AS a, 2 AS b) AS t",
read={
"": "SELECT a, b FROM (VALUES (1, 2)) AS t (a, b)",
},
)
self.validate_all(
'SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS "t" (a, b)',
write={
"redshift": 'SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS "t"',
'SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS "t"',
read={
"": 'SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS "t" (a, b)',
},
)
self.validate_all(
"SELECT a, b FROM (VALUES (1, 2), (3, 4), (5, 6), (7, 8)) AS t (a, b)",
write={
"redshift": "SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4 UNION ALL SELECT 5, 6 UNION ALL SELECT 7, 8) AS t",
"SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4 UNION ALL SELECT 5, 6 UNION ALL SELECT 7, 8) AS t",
read={
"": "SELECT a, b FROM (VALUES (1, 2), (3, 4), (5, 6), (7, 8)) AS t (a, b)",
},
)
self.validate_all(
"INSERT INTO t(a) VALUES (1), (2), (3)",
write={
"redshift": "INSERT INTO t (a) VALUES (1), (2), (3)",
"INSERT INTO t (a, b) SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS t",
read={
"": "INSERT INTO t(a, b) SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS t (a, b)",
},
)
self.validate_all(
"INSERT INTO t(a, b) SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS t (a, b)",
write={
"redshift": "INSERT INTO t (a, b) SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS t",
},
)
self.validate_all(
"INSERT INTO t(a, b) VALUES (1, 2), (3, 4)",
write={
"redshift": "INSERT INTO t (a, b) VALUES (1, 2), (3, 4)",
},
)
self.validate_identity(
'SELECT * FROM (VALUES (1)) AS "t"(a)',
'''SELECT
*
FROM (
SELECT
1 AS a
) AS "t"''',
pretty=True,
)
def test_create_table_like(self):
self.validate_identity(
@ -532,3 +513,26 @@ FROM (
"redshift": "CREATE OR REPLACE VIEW v1 AS SELECT cola, colb FROM t1 WITH NO SCHEMA BINDING",
},
)
def test_column_unnesting(self):
ast = parse_one("SELECT * FROM t.t JOIN t.c1 ON c1.c2 = t.c3", read="redshift")
ast.args["from"].this.assert_is(exp.Table)
ast.args["joins"][0].this.assert_is(exp.Table)
self.assertEqual(ast.sql("redshift"), "SELECT * FROM t.t JOIN t.c1 ON c1.c2 = t.c3")
ast = parse_one("SELECT * FROM t AS t CROSS JOIN t.c1", read="redshift")
ast.args["from"].this.assert_is(exp.Table)
ast.args["joins"][0].this.assert_is(exp.Column)
self.assertEqual(ast.sql("redshift"), "SELECT * FROM t AS t CROSS JOIN t.c1")
ast = parse_one(
"SELECT * FROM x AS a, a.b AS c, c.d.e AS f, f.g.h.i.j.k AS l", read="redshift"
)
joins = ast.args["joins"]
ast.args["from"].this.assert_is(exp.Table)
joins[0].this.this.assert_is(exp.Column)
joins[1].this.this.assert_is(exp.Column)
joins[2].this.this.assert_is(exp.Dot)
self.assertEqual(
ast.sql("redshift"), "SELECT * FROM x AS a, a.b AS c, c.d.e AS f, f.g.h.i.j.k AS l"
)

18
tests/dialects/test_snowflake.py
View file

@ -84,6 +84,10 @@ WHERE
self.validate_identity(
"SELECT a FROM test PIVOT(SUM(x) FOR y IN ('z', 'q')) AS x TABLESAMPLE (0.1)"
)
self.validate_identity(
"value:values::string",
"CAST(GET_PATH(value, 'values') AS TEXT)",
)
self.validate_identity(
"""SELECT GET_PATH(PARSE_JSON('{"y": [{"z": 1}]}'), 'y[0]:z')""",
"""SELECT GET_PATH(PARSE_JSON('{"y": [{"z": 1}]}'), 'y[0].z')""",
@ -462,7 +466,7 @@ WHERE
"DIV0(foo, bar)",
write={
"snowflake": "IFF(bar = 0, 0, foo / bar)",
"sqlite": "CASE WHEN bar = 0 THEN 0 ELSE CAST(foo AS REAL) / bar END",
"sqlite": "IIF(bar = 0, 0, CAST(foo AS REAL) / bar)",
"presto": "IF(bar = 0, 0, CAST(foo AS DOUBLE) / bar)",
"spark": "IF(bar = 0, 0, foo / bar)",
"hive": "IF(bar = 0, 0, foo / bar)",
@ -473,7 +477,7 @@ WHERE
"ZEROIFNULL(foo)",
write={
"snowflake": "IFF(foo IS NULL, 0, foo)",
"sqlite": "CASE WHEN foo IS NULL THEN 0 ELSE foo END",
"sqlite": "IIF(foo IS NULL, 0, foo)",
"presto": "IF(foo IS NULL, 0, foo)",
"spark": "IF(foo IS NULL, 0, foo)",
"hive": "IF(foo IS NULL, 0, foo)",
@ -484,7 +488,7 @@ WHERE
"NULLIFZERO(foo)",
write={
"snowflake": "IFF(foo = 0, NULL, foo)",
"sqlite": "CASE WHEN foo = 0 THEN NULL ELSE foo END",
"sqlite": "IIF(foo = 0, NULL, foo)",
"presto": "IF(foo = 0, NULL, foo)",
"spark": "IF(foo = 0, NULL, foo)",
"hive": "IF(foo = 0, NULL, foo)",
@ -1513,6 +1517,10 @@ MATCH_RECOGNIZE (
self.validate_identity("SHOW COLUMNS IN VIEW")
self.validate_identity("SHOW COLUMNS LIKE '_foo%' IN VIEW dt_test")
self.validate_identity("SHOW USERS")
self.validate_identity("SHOW TERSE USERS")
self.validate_identity("SHOW USERS LIKE '_foo%' STARTS WITH 'bar' LIMIT 5 FROM 'baz'")
ast = parse_one("SHOW COLUMNS LIKE '_testing%' IN dt_test", read="snowflake")
table = ast.find(exp.Table)
literal = ast.find(exp.Literal)
@ -1536,6 +1544,10 @@ MATCH_RECOGNIZE (
table = ast.find(exp.Table)
self.assertEqual(table.sql(dialect="snowflake"), "db1.schema1")
users_exp = self.validate_identity("SHOW USERS")
self.assertTrue(isinstance(users_exp, exp.Show))
self.assertEqual(users_exp.this, "USERS")
def test_swap(self):
ast = parse_one("ALTER TABLE a SWAP WITH b", read="snowflake")
assert isinstance(ast, exp.AlterTable)

26
tests/dialects/test_spark.py
View file

@ -93,11 +93,12 @@ TBLPROPERTIES (
'x'='1'
)""",
"spark": """CREATE TABLE blah (
col_a INT
col_a INT,
date STRING
)
COMMENT 'Test comment: blah'
PARTITIONED BY (
date STRING
date
)
USING ICEBERG
TBLPROPERTIES (
@ -125,13 +126,6 @@ TBLPROPERTIES (
"spark": "ALTER TABLE StudentInfo DROP COLUMNS (LastName, DOB)",
},
)
self.validate_all(
"CREATE TABLE x USING ICEBERG PARTITIONED BY (MONTHS(y)) LOCATION 's3://z'",
identify=True,
write={
"spark": "CREATE TABLE `x` USING ICEBERG PARTITIONED BY (MONTHS(`y`)) LOCATION 's3://z'",
},
)
def test_to_date(self):
self.validate_all(
@ -256,6 +250,14 @@ TBLPROPERTIES (
self.validate_identity("TRIM(LEADING 'SL' FROM 'SSparkSQLS')")
self.validate_identity("TRIM(TRAILING 'SL' FROM 'SSparkSQLS')")
self.validate_identity("SPLIT(str, pattern, lim)")
self.validate_identity(
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL 23 HOUR + 59 MINUTE + 59 SECONDS",
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL '23' HOUR + INTERVAL '59' MINUTE + INTERVAL '59' SECONDS",
)
self.validate_identity(
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL '23' HOUR + '59' MINUTE + '59' SECONDS",
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL '23' HOUR + INTERVAL '59' MINUTE + INTERVAL '59' SECONDS",
)
self.validate_identity(
"SELECT INTERVAL '5' HOURS '30' MINUTES '5' SECONDS '6' MILLISECONDS '7' MICROSECONDS",
"SELECT INTERVAL '5' HOURS + INTERVAL '30' MINUTES + INTERVAL '5' SECONDS + INTERVAL '6' MILLISECONDS + INTERVAL '7' MICROSECONDS",
@ -616,12 +618,6 @@ TBLPROPERTIES (
},
)
def test_iif(self):
self.validate_all(
"SELECT IIF(cond, 'True', 'False')",
write={"spark": "SELECT IF(cond, 'True', 'False')"},
)
def test_bool_or(self):
self.validate_all(
"SELECT a, LOGICAL_OR(b) FROM table GROUP BY a",

15
tests/dialects/test_tableau.py
View file

@ -5,6 +5,21 @@ class TestTableau(Validator):
dialect = "tableau"
def test_tableau(self):
self.validate_all(
"[x]",
write={
"hive": "`x`",
"tableau": "[x]",
},
)
self.validate_all(
'"x"',
write={
"hive": "'x'",
"tableau": "'x'",
},
)
self.validate_all(
"IF x = 'a' THEN y ELSE NULL END",
read={

22
tests/dialects/test_tsql.py
View file

@ -28,6 +28,14 @@ class TestTSQL(Validator):
self.validate_identity("1 AND true", "1 <> 0 AND (1 = 1)")
self.validate_identity("CAST(x AS int) OR y", "CAST(x AS INTEGER) <> 0 OR y <> 0")
self.validate_all(
"SELECT IIF(cond <> 0, 'True', 'False')",
read={
"spark": "SELECT IF(cond, 'True', 'False')",
"sqlite": "SELECT IIF(cond, 'True', 'False')",
"tsql": "SELECT IIF(cond <> 0, 'True', 'False')",
},
)
self.validate_all(
"SELECT TRIM(BOTH 'a' FROM a)",
read={
@ -1302,20 +1310,6 @@ WHERE
},
)
def test_iif(self):
self.validate_identity(
"SELECT IF(cond, 'True', 'False')", "SELECT IIF(cond <> 0, 'True', 'False')"
)
self.validate_identity(
"SELECT IIF(cond, 'True', 'False')", "SELECT IIF(cond <> 0, 'True', 'False')"
)
self.validate_all(
"SELECT IIF(cond, 'True', 'False');",
write={
"spark": "SELECT IF(cond, 'True', 'False')",
},
)
def test_lateral_subquery(self):
self.validate_all(
"SELECT x.a, x.b, t.v, t.y FROM x CROSS APPLY (SELECT v, y FROM t) t(v, y)",

3
tests/fixtures/identity.sql vendored
View file

@ -847,3 +847,6 @@ WITH use(use) AS (SELECT 1) SELECT use FROM use
SELECT recursive FROM t
SELECT (ROW_NUMBER() OVER (PARTITION BY user ORDER BY date ASC) - ROW_NUMBER() OVER (PARTITION BY user, segment ORDER BY date ASC)) AS group_id FROM example_table
CAST(foo AS BPCHAR)
values
SELECT values
SELECT values AS values FROM t WHERE values + 1 > 3

12
tests/fixtures/optimizer/canonicalize.sql vendored
View file

@ -96,3 +96,15 @@ DATE_TRUNC('DAY', CAST('2023-01-01' AS DATE));
DATEDIFF('2023-01-01', '2023-01-02', DAY);
DATEDIFF(CAST('2023-01-01' AS DATETIME), CAST('2023-01-02' AS DATETIME), DAY);
--------------------------------------
-- Remove redundant casts
--------------------------------------
CAST(CAST('2023-01-01' AS DATE) AS DATE);
CAST('2023-01-01' AS DATE);
CAST(DATE_TRUNC('YEAR', CAST('2023-01-01' AS DATE)) AS DATE);
DATE_TRUNC('YEAR', CAST('2023-01-01' AS DATE));
DATE(DATE_TRUNC('YEAR', CAST("x" AS DATE)));
DATE_TRUNC('YEAR', CAST("x" AS DATE));

6
tests/fixtures/optimizer/merge_subqueries.sql vendored
View file

@ -218,6 +218,7 @@ with t1 as (
ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num
FROM
x
ORDER BY x.a, x.b, row_num
)
SELECT
t1.a,
@ -226,7 +227,7 @@ FROM
t1
WHERE
row_num = 1;
WITH t1 AS (SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x) SELECT t1.a AS a, t1.b AS b FROM t1 AS t1 WHERE t1.row_num = 1;
WITH t1 AS (SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x ORDER BY x.a, x.b, row_num) SELECT t1.a AS a, t1.b AS b FROM t1 AS t1 WHERE t1.row_num = 1;
# title: Test preventing merge of window expressions join clause
with t1 as (
@ -301,6 +302,7 @@ with t1 as (
ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num
FROM
x
ORDER BY x.a, x.b, row_num
)
SELECT
t1.a,
@ -308,7 +310,7 @@ SELECT
t1.row_num
FROM
t1;
SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x;
SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x ORDER BY x.a, x.b, row_num;
# title: Don't merge window functions, inner table is aliased in outer query
with t1 as (

32
tests/fixtures/optimizer/qualify_columns.sql vendored
View file

@ -208,14 +208,14 @@ SELECT x.a AS a, y.c AS c FROM x AS x, y AS y;
--------------------------------------
-- Unions
--------------------------------------
SELECT a FROM x UNION SELECT a FROM x;
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x;
SELECT a FROM x UNION SELECT a FROM x ORDER BY a;
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x ORDER BY a;
SELECT a FROM x UNION SELECT a FROM x UNION SELECT a FROM x;
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x;
SELECT a FROM x UNION SELECT a FROM x UNION SELECT a FROM x ORDER BY a;
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x ORDER BY a;
SELECT a FROM (SELECT a FROM x UNION SELECT a FROM x);
SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x) AS _q_0;
SELECT a FROM (SELECT a FROM x UNION SELECT a FROM x) ORDER BY a;
SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x) AS _q_0 ORDER BY a;
--------------------------------------
-- Subqueries
@ -318,8 +318,8 @@ WITH z AS (SELECT x.a AS a FROM x AS x) SELECT z.a AS a FROM z AS z;
WITH z AS (SELECT a FROM x), q AS (SELECT * FROM z) SELECT * FROM q;
WITH z AS (SELECT x.a AS a FROM x AS x), q AS (SELECT z.a AS a FROM z AS z) SELECT q.a AS a FROM q AS q;
WITH z AS (SELECT * FROM x) SELECT * FROM z UNION SELECT * FROM z;
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x) SELECT z.a AS a, z.b AS b FROM z AS z UNION SELECT z.a AS a, z.b AS b FROM z AS z;
WITH z AS (SELECT * FROM x) SELECT * FROM z UNION SELECT * FROM z ORDER BY a, b;
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x) SELECT z.a AS a, z.b AS b FROM z AS z UNION SELECT z.a AS a, z.b AS b FROM z AS z ORDER BY a, b;
WITH z AS (SELECT * FROM x), q AS (SELECT b FROM z) SELECT b FROM q;
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x), q AS (SELECT z.b AS b FROM z AS z) SELECT q.b AS b FROM q AS q;
@ -359,8 +359,8 @@ SELECT x.b AS b FROM x AS x;
SELECT * EXCEPT (a, b) FROM x;
SELECT * EXCEPT (x.a, x.b) FROM x AS x;
SELECT COALESCE(t1.a, '') AS a, t2.* EXCEPT (a) FROM x AS t1, x AS t2;
SELECT COALESCE(t1.a, '') AS a, t2.b AS b FROM x AS t1, x AS t2;
SELECT COALESCE(CAST(t1.a AS VARCHAR), '') AS a, t2.* EXCEPT (a) FROM x AS t1, x AS t2;
SELECT COALESCE(CAST(t1.a AS VARCHAR), '') AS a, t2.b AS b FROM x AS t1, x AS t2;
--------------------------------------
-- Using
@ -468,8 +468,8 @@ select * from unnest ([1, 2]) as x with offset as y;
SELECT x AS x, y AS y FROM UNNEST([1, 2]) AS x WITH OFFSET AS y;
# dialect: presto
SELECT x.a, i.b FROM x CROSS JOIN UNNEST(SPLIT(b, ',')) AS i(b);
SELECT x.a AS a, i.b AS b FROM x AS x CROSS JOIN UNNEST(SPLIT(x.b, ',')) AS i(b);
SELECT x.a, i.b FROM x CROSS JOIN UNNEST(SPLIT(CAST(b AS VARCHAR), ',')) AS i(b);
SELECT x.a AS a, i.b AS b FROM x AS x CROSS JOIN UNNEST(SPLIT(CAST(x.b AS VARCHAR), ',')) AS i(b);
# execute: false
SELECT c FROM (SELECT 1 a) AS x LATERAL VIEW EXPLODE(a) AS c;
@ -487,16 +487,16 @@ SELECT t.c1 AS c1, t.c3 AS c3 FROM FOO(bar) AS t(c1, c2, c3);
-- Window functions
--------------------------------------
# title: ORDER BY in window function
SELECT a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY b ORDER BY a) AS row_num FROM x;
SELECT x.a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a) AS row_num FROM x AS x;
SELECT a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY b ORDER BY a) AS row_num FROM x ORDER BY a, row_num;
SELECT x.a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a) AS row_num FROM x AS x ORDER BY a, row_num;
# dialect: bigquery
SELECT ROW_NUMBER() OVER (PARTITION BY a ORDER BY b) AS row_num FROM x QUALIFY row_num = 1;
SELECT ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.b) AS row_num FROM x AS x QUALIFY ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.b) = 1;
# dialect: bigquery
SELECT x.b, x.a FROM x LEFT JOIN y ON x.b = y.b QUALIFY ROW_NUMBER() OVER(PARTITION BY x.b ORDER BY x.a DESC) = 1;
SELECT x.b AS b, x.a AS a FROM x AS x LEFT JOIN y AS y ON x.b = y.b QUALIFY ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a DESC) = 1;
SELECT x.b, x.a FROM x LEFT JOIN y ON x.b = y.b QUALIFY ROW_NUMBER() OVER(PARTITION BY x.b ORDER BY x.a DESC) = 1 ORDER BY x.b, x.a;
SELECT x.b AS b, x.a AS a FROM x AS x LEFT JOIN y AS y ON x.b = y.b QUALIFY ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a DESC) = 1 ORDER BY x.b, x.a;
SELECT * FROM x QUALIFY COUNT(a) OVER (PARTITION BY b) > 1;
SELECT x.a AS a, x.b AS b FROM x AS x QUALIFY COUNT(x.a) OVER (PARTITION BY x.b) > 1;

38
tests/fixtures/optimizer/tpc-h/tpc-h.sql vendored
View file

@ -15,7 +15,7 @@ select
from
lineitem
where
l_shipdate <= date '1998-12-01' - interval '90' day
CAST(l_shipdate AS DATE) <= date '1998-12-01' - interval '90' day
group by
l_returnflag,
l_linestatus
@ -218,8 +218,8 @@ select
from
orders
where
o_orderdate >= date '1993-07-01'
and o_orderdate < date '1993-07-01' + interval '3' month
CAST(o_orderdate AS DATE) >= date '1993-07-01'
and CAST(o_orderdate AS DATE) < date '1993-07-01' + interval '3' month
and exists (
select
*
@ -278,8 +278,8 @@ where
and s_nationkey = n_nationkey
and n_regionkey = r_regionkey
and r_name = 'ASIA'
and o_orderdate >= date '1994-01-01'
and o_orderdate < date '1994-01-01' + interval '1' year
and CAST(o_orderdate AS DATE) >= date '1994-01-01'
and CAST(o_orderdate AS DATE) < date '1994-01-01' + interval '1' year
group by
n_name
order by
@ -316,8 +316,8 @@ select
from
lineitem
where
l_shipdate >= date '1994-01-01'
and l_shipdate < date '1994-01-01' + interval '1' year
CAST(l_shipdate AS DATE) >= date '1994-01-01'
and CAST(l_shipdate AS DATE) < date '1994-01-01' + interval '1' year
and l_discount between 0.06 - 0.01 and 0.06 + 0.01
and l_quantity < 24;
SELECT
@ -362,7 +362,7 @@ from
(n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
)
and l_shipdate between date '1995-01-01' and date '1996-12-31'
and CAST(l_shipdate AS DATE) between date '1995-01-01' and date '1996-12-31'
) as shipping
group by
supp_nation,
@ -446,7 +446,7 @@ from
and n1.n_regionkey = r_regionkey
and r_name = 'AMERICA'
and s_nationkey = n2.n_nationkey
and o_orderdate between date '1995-01-01' and date '1996-12-31'
and CAST(o_orderdate AS DATE) between date '1995-01-01' and date '1996-12-31'
and p_type = 'ECONOMY ANODIZED STEEL'
) as all_nations
group by
@ -574,8 +574,8 @@ from
where
c_custkey = o_custkey
and l_orderkey = o_orderkey
and o_orderdate >= date '1993-10-01'
and o_orderdate < date '1993-10-01' + interval '3' month
and CAST(o_orderdate AS DATE) >= date '1993-10-01'
and CAST(o_orderdate AS DATE) < date '1993-10-01' + interval '3' month
and l_returnflag = 'R'
and c_nationkey = n_nationkey
group by
@ -714,8 +714,8 @@ where
and l_shipmode in ('MAIL', 'SHIP')
and l_commitdate < l_receiptdate
and l_shipdate < l_commitdate
and l_receiptdate >= date '1994-01-01'
and l_receiptdate < date '1994-01-01' + interval '1' year
and CAST(l_receiptdate AS DATE) >= date '1994-01-01'
and CAST(l_receiptdate AS DATE) < date '1994-01-01' + interval '1' year
group by
l_shipmode
order by
@ -813,8 +813,8 @@ from
part
where
l_partkey = p_partkey
and l_shipdate >= date '1995-09-01'
and l_shipdate < date '1995-09-01' + interval '1' month;
and CAST(l_shipdate AS DATE) >= date '1995-09-01'
and CAST(l_shipdate AS DATE) < date '1995-09-01' + interval '1' month;
SELECT
100.00 * SUM(
CASE
@ -844,8 +844,8 @@ with revenue (supplier_no, total_revenue) as (
from
lineitem
where
l_shipdate >= date '1996-01-01'
and l_shipdate < date '1996-01-01' + interval '3' month
CAST(l_shipdate AS DATE) >= date '1996-01-01'
and CAST(l_shipdate AS DATE) < date '1996-01-01' + interval '3' month
group by
l_suppkey)
select
@ -1223,8 +1223,8 @@ where
where
l_partkey = ps_partkey
and l_suppkey = ps_suppkey
and l_shipdate >= date '1994-01-01'
and l_shipdate < date '1994-01-01' + interval '1' year
and CAST(l_shipdate AS DATE) >= date '1994-01-01'
and CAST(l_shipdate AS DATE) < date '1994-01-01' + interval '1' year
)
)
and s_nationkey = n_nationkey

9
tests/test_expressions.py
View file

@ -243,6 +243,15 @@ class TestExpressions(unittest.TestCase):
'SELECT * FROM a1 AS a /* a */, b.a /* b */, c.a2 /* c.a */, d2 /* d.a */ CROSS JOIN e.a CROSS JOIN "F" /* f-F.A */ CROSS JOIN g1.a /* g */',
)
self.assertEqual(
exp.replace_tables(
parse_one("select * from example.table", dialect="bigquery"),
{"example.table": "`my-project.example.table`"},
dialect="bigquery",
).sql(),
'SELECT * FROM "my-project".example.table /* example.table */',
)
def test_expand(self):
self.assertEqual(
exp.expand(

33
tests/test_lineage.py
View file

@ -95,7 +95,7 @@ class TestLineage(unittest.TestCase):
downstream.source.sql(),
"SELECT x.a AS a FROM x AS x",
)
self.assertEqual(downstream.alias, "")
self.assertEqual(downstream.alias, "z")
def test_lineage_source_with_star(self) -> None:
node = lineage(
@ -153,7 +153,7 @@ class TestLineage(unittest.TestCase):
downstream = downstream.downstream[0]
self.assertEqual(downstream.source.sql(), "(VALUES (1), (2)) AS t(a)")
self.assertEqual(downstream.expression.sql(), "a")
self.assertEqual(downstream.alias, "")
self.assertEqual(downstream.alias, "y")
def test_lineage_cte_name_appears_in_schema(self) -> None:
schema = {"a": {"b": {"t1": {"c1": "int"}, "t2": {"c2": "int"}}}}
@ -284,6 +284,35 @@ class TestLineage(unittest.TestCase):
self.assertEqual(downstream_b.name, "0")
self.assertEqual(downstream_b.source.sql(), "SELECT * FROM catalog.db.table_b AS table_b")
def test_lineage_source_union(self) -> None:
query = "SELECT x, created_at FROM dataset;"
node = lineage(
"x",
query,
sources={
"dataset": """
SELECT *
FROM catalog.db.table_a
UNION
SELECT *
FROM catalog.db.table_b
"""
},
)
self.assertEqual(node.name, "x")
downstream_a = node.downstream[0]
self.assertEqual(downstream_a.name, "0")
self.assertEqual(downstream_a.alias, "dataset")
self.assertEqual(downstream_a.source.sql(), "SELECT * FROM catalog.db.table_a AS table_a")
downstream_b = node.downstream[1]
self.assertEqual(downstream_b.name, "0")
self.assertEqual(downstream_b.alias, "dataset")
self.assertEqual(downstream_b.source.sql(), "SELECT * FROM catalog.db.table_b AS table_b")
def test_select_star(self) -> None:
node = lineage("x", "SELECT x from (SELECT * from table_a)")

6
tests/test_optimizer.py
View file

@ -874,6 +874,12 @@ FROM READ_CSV('tests/fixtures/optimizer/tpc-h/nation.csv.gz', 'delimiter', '|')
self.assertEqual(case_ifs_expr.type.this, exp.DataType.Type.VARCHAR)
self.assertEqual(case_ifs_expr.args["true"].type.this, exp.DataType.Type.VARCHAR)
timestamp = annotate_types(parse_one("TIMESTAMP(x)"))
self.assertEqual(timestamp.type.this, exp.DataType.Type.TIMESTAMP)
timestamptz = annotate_types(parse_one("TIMESTAMP(x)", read="bigquery"))
self.assertEqual(timestamptz.type.this, exp.DataType.Type.TIMESTAMPTZ)
def test_unknown_annotation(self):
schema = {"x": {"cola": "VARCHAR"}}
sql = "SELECT x.cola + SOME_ANONYMOUS_FUNC(x.cola) AS col FROM x AS x"

13
tests/test_parser.py
View file

@ -839,3 +839,16 @@ class TestParser(unittest.TestCase):
),
)
self.assertEqual(ast.sql(dialect=dialect), "CREATE SCHEMA catalog.schema")
def test_values_as_identifier(self):
sql = "SELECT values FROM t WHERE values + 1 > x"
for dialect in (
"bigquery",
"clickhouse",
"duckdb",
"postgres",
"redshift",
"snowflake",
):
with self.subTest(dialect):
self.assertEqual(parse_one(sql, dialect=dialect).sql(dialect=dialect), sql)

3
tests/test_transpile.py
View file

@ -554,11 +554,12 @@ FROM base""",
self.validate(
"WITH A(filter) AS (VALUES 1, 2, 3) SELECT * FROM A WHERE filter >= 2",
"WITH A(filter) AS (VALUES (1), (2), (3)) SELECT * FROM A WHERE filter >= 2",
read="presto",
)
self.validate(
"SELECT BOOL_OR(a > 10) FROM (VALUES 1, 2, 15) AS T(a)",
"SELECT BOOL_OR(a > 10) FROM (VALUES (1), (2), (15)) AS T(a)",
write="presto",
read="presto",
)
def test_alter(self):