Merging upstream version 21.1.1.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
4e41aa0bbb
commit
bf03050a25
GPG key ID:
FBB4F0E80A80222F
91 changed files with 49165 additions and 47854 deletions
25
CHANGELOG.md
25
CHANGELOG.md
|
|
@ -1,6 +1,30 @@
|
||||||
Changelog
|
Changelog
|
||||||
=========
|
=========
|
||||||
|
|
||||||
|
## [v21.1.0] - 2024-02-12
|
||||||
|
### :sparkles: New Features
|
||||||
|
- [`e71d489`](https://github.com/tobymao/sqlglot/commit/e71d4899e6744812fdefc2704c66bbd6043b5bc9) - add array and tuple helpers *(commit by [@tobymao](https://github.com/tobymao))*
|
||||||
|
- [`876e075`](https://github.com/tobymao/sqlglot/commit/876e07580bb2de06b587fc8ad40eb67604ae8507) - **postgres**: root operator closes [#2940](https://github.com/tobymao/sqlglot/pull/2940) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||||
|
- [`e731276`](https://github.com/tobymao/sqlglot/commit/e731276dd5490a7d294430e0887eebf19e16d28f) - **snowflake**: add support for SHOW USERS *(PR [#2948](https://github.com/tobymao/sqlglot/pull/2948) by [@DanCardin](https://github.com/DanCardin))*
|
||||||
|
- [`b9d4468`](https://github.com/tobymao/sqlglot/commit/b9d44688c2b785212db635f121b686df02e2dec9) - **tableau**: identifier and quotes closes [#2950](https://github.com/tobymao/sqlglot/pull/2950) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||||
|
- [`f8d9dbf`](https://github.com/tobymao/sqlglot/commit/f8d9dbf6744f95bf4b7517e8bcc35dd3a6f70c5d) - **sqlite**: add support for IIF *(PR [#2951](https://github.com/tobymao/sqlglot/pull/2951) by [@georgesittas](https://github.com/georgesittas))*
|
||||||
|
- [`b755551`](https://github.com/tobymao/sqlglot/commit/b7555516c6bf038dc39c4bba2b243839ceb6e3b5) - **clickhouse**: add basic support for system statement *(PR [#2953](https://github.com/tobymao/sqlglot/pull/2953) by [@GaliFFun](https://github.com/GaliFFun))*
|
||||||
|
|
||||||
|
### :bug: Bug Fixes
|
||||||
|
- [`844018b`](https://github.com/tobymao/sqlglot/commit/844018b8d3a3398d746fdc04c966c7e19d311998) - explode_outer to unnest closes [#2941](https://github.com/tobymao/sqlglot/pull/2941) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||||
|
- [`159da45`](https://github.com/tobymao/sqlglot/commit/159da4523d6eb3ca6853d631bb98dc8f13c7b0fb) - posexplode_outer to unnest *(PR [#2942](https://github.com/tobymao/sqlglot/pull/2942) by [@chelsea-lin](https://github.com/chelsea-lin))*
|
||||||
|
- [`76d6634`](https://github.com/tobymao/sqlglot/commit/76d66340e566bd9fa8c783f5d311101eb2e80480) - **spark**: CREATE TABLE ... PARTITIONED BY fixes *(PR [#2937](https://github.com/tobymao/sqlglot/pull/2937) by [@barakalon](https://github.com/barakalon))*
|
||||||
|
- [`d07ddf9`](https://github.com/tobymao/sqlglot/commit/d07ddf9b460c1b6f672fda4f34dc9231419e6c9d) - **optimizer**: remove redundant casts *(PR [#2945](https://github.com/tobymao/sqlglot/pull/2945) by [@barakalon](https://github.com/barakalon))*
|
||||||
|
- [`b70a394`](https://github.com/tobymao/sqlglot/commit/b70a394222bf209298026fd100f6b9498acf9fff) - if doesn't support different types *(commit by [@tobymao](https://github.com/tobymao))*
|
||||||
|
- [`6a988e0`](https://github.com/tobymao/sqlglot/commit/6a988e0160022d33623cd036bf84bb0b222c9062) - **bigquery**: fix annotation of timestamp(x) *(PR [#2946](https://github.com/tobymao/sqlglot/pull/2946) by [@georgesittas](https://github.com/georgesittas))*
|
||||||
|
- [`78e6d0d`](https://github.com/tobymao/sqlglot/commit/78e6d0de83efbff1d3b61c8550db56c1819f7c22) - **optimizer**: qualify_columns optimizations for wide tables *(PR [#2955](https://github.com/tobymao/sqlglot/pull/2955) by [@barakalon](https://github.com/barakalon))*
|
||||||
|
- [`c20cc70`](https://github.com/tobymao/sqlglot/commit/c20cc70dfc7f6395af157521c7e99074d697beb4) - **redshift**: don't assume Table is an unnested Column if Join has a predicate *(PR [#2956](https://github.com/tobymao/sqlglot/pull/2956) by [@georgesittas](https://github.com/georgesittas))*
|
||||||
|
- :arrow_lower_right: *fixes issue [#2952](https://github.com/tobymao/sqlglot/issues/2952) opened by [@vidit-wisdom](https://github.com/vidit-wisdom)*
|
||||||
|
|
||||||
|
### :wrench: Chores
|
||||||
|
- [`c4524ce`](https://github.com/tobymao/sqlglot/commit/c4524ce1e6a85e16db7ea0289116d0160732dc51) - fix unit test *(commit by [@tobymao](https://github.com/tobymao))*
|
||||||
|
|
||||||
|
|
||||||
## [v21.0.2] - 2024-02-08
|
## [v21.0.2] - 2024-02-08
|
||||||
### :sparkles: New Features
|
### :sparkles: New Features
|
||||||
- [`1842c96`](https://github.com/tobymao/sqlglot/commit/1842c96611cadb0227dd3ce8f42457679ab0e08b) - **clickhouse**: add support for LIMIT BY clause *(PR [#2926](https://github.com/tobymao/sqlglot/pull/2926) by [@georgesittas](https://github.com/georgesittas))*
|
- [`1842c96`](https://github.com/tobymao/sqlglot/commit/1842c96611cadb0227dd3ce8f42457679ab0e08b) - **clickhouse**: add support for LIMIT BY clause *(PR [#2926](https://github.com/tobymao/sqlglot/pull/2926) by [@georgesittas](https://github.com/georgesittas))*
|
||||||
|
|
@ -2594,3 +2618,4 @@ Changelog
|
||||||
[v21.0.0]: https://github.com/tobymao/sqlglot/compare/v20.11.0...v21.0.0
|
[v21.0.0]: https://github.com/tobymao/sqlglot/compare/v20.11.0...v21.0.0
|
||||||
[v21.0.1]: https://github.com/tobymao/sqlglot/compare/v21.0.0...v21.0.1
|
[v21.0.1]: https://github.com/tobymao/sqlglot/compare/v21.0.0...v21.0.1
|
||||||
[v21.0.2]: https://github.com/tobymao/sqlglot/compare/v21.0.1...v21.0.2
|
[v21.0.2]: https://github.com/tobymao/sqlglot/compare/v21.0.1...v21.0.2
|
||||||
|
[v21.1.0]: https://github.com/tobymao/sqlglot/compare/v21.0.2...v21.1.0
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -76,8 +76,8 @@
|
||||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
|
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
|
||||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
|
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
|
||||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
|
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
|
||||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'21.0.3.dev0'</span>
|
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'21.1.0'</span>
|
||||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">21</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="s1">'dev0'</span><span class="p">)</span>
|
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">21</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -97,7 +97,7 @@
|
||||||
<section id="version">
|
<section id="version">
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">version</span><span class="annotation">: str</span> =
|
<span class="name">version</span><span class="annotation">: str</span> =
|
||||||
<span class="default_value">'21.0.3.dev0'</span>
|
<span class="default_value">'21.1.0'</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
@ -109,7 +109,7 @@
|
||||||
<section id="version_tuple">
|
<section id="version_tuple">
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">version_tuple</span><span class="annotation">: object</span> =
|
<span class="name">version_tuple</span><span class="annotation">: object</span> =
|
||||||
<span class="default_value">(21, 0, 3, 'dev0')</span>
|
<span class="default_value">(21, 1, 0)</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
|
|
@ -607,126 +607,124 @@
|
||||||
</span><span id="SparkSession-71"><a href="#SparkSession-71"><span class="linenos"> 71</span></a> <span class="n">column_mapping</span> <span class="o">=</span> <span class="p">{</span><span class="sa">f</span><span class="s2">"_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">:</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)}</span>
|
</span><span id="SparkSession-71"><a href="#SparkSession-71"><span class="linenos"> 71</span></a> <span class="n">column_mapping</span> <span class="o">=</span> <span class="p">{</span><span class="sa">f</span><span class="s2">"_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">:</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)}</span>
|
||||||
</span><span id="SparkSession-72"><a href="#SparkSession-72"><span class="linenos"> 72</span></a>
|
</span><span id="SparkSession-72"><a href="#SparkSession-72"><span class="linenos"> 72</span></a>
|
||||||
</span><span id="SparkSession-73"><a href="#SparkSession-73"><span class="linenos"> 73</span></a> <span class="n">data_expressions</span> <span class="o">=</span> <span class="p">[</span>
|
</span><span id="SparkSession-73"><a href="#SparkSession-73"><span class="linenos"> 73</span></a> <span class="n">data_expressions</span> <span class="o">=</span> <span class="p">[</span>
|
||||||
</span><span id="SparkSession-74"><a href="#SparkSession-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Tuple</span><span class="p">(</span>
|
</span><span id="SparkSession-74"><a href="#SparkSession-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">tuple_</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-75"><a href="#SparkSession-75"><span class="linenos"> 75</span></a> <span class="n">expressions</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span>
|
</span><span id="SparkSession-75"><a href="#SparkSession-75"><span class="linenos"> 75</span></a> <span class="o">*</span><span class="nb">map</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-76"><a href="#SparkSession-76"><span class="linenos"> 76</span></a> <span class="nb">map</span><span class="p">(</span>
|
</span><span id="SparkSession-76"><a href="#SparkSession-76"><span class="linenos"> 76</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-77"><a href="#SparkSession-77"><span class="linenos"> 77</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
</span><span id="SparkSession-77"><a href="#SparkSession-77"><span class="linenos"> 77</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
|
||||||
</span><span id="SparkSession-78"><a href="#SparkSession-78"><span class="linenos"> 78</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
|
</span><span id="SparkSession-78"><a href="#SparkSession-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession-79"><a href="#SparkSession-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
|
</span><span id="SparkSession-79"><a href="#SparkSession-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession-80"><a href="#SparkSession-80"><span class="linenos"> 80</span></a> <span class="p">)</span>
|
</span><span id="SparkSession-80"><a href="#SparkSession-80"><span class="linenos"> 80</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
|
||||||
</span><span id="SparkSession-81"><a href="#SparkSession-81"><span class="linenos"> 81</span></a> <span class="p">)</span>
|
</span><span id="SparkSession-81"><a href="#SparkSession-81"><span class="linenos"> 81</span></a> <span class="p">]</span>
|
||||||
</span><span id="SparkSession-82"><a href="#SparkSession-82"><span class="linenos"> 82</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
|
</span><span id="SparkSession-82"><a href="#SparkSession-82"><span class="linenos"> 82</span></a>
|
||||||
</span><span id="SparkSession-83"><a href="#SparkSession-83"><span class="linenos"> 83</span></a> <span class="p">]</span>
|
</span><span id="SparkSession-83"><a href="#SparkSession-83"><span class="linenos"> 83</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
|
||||||
</span><span id="SparkSession-84"><a href="#SparkSession-84"><span class="linenos"> 84</span></a>
|
</span><span id="SparkSession-84"><a href="#SparkSession-84"><span class="linenos"> 84</span></a> <span class="p">(</span>
|
||||||
</span><span id="SparkSession-85"><a href="#SparkSession-85"><span class="linenos"> 85</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
|
</span><span id="SparkSession-85"><a href="#SparkSession-85"><span class="linenos"> 85</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
||||||
</span><span id="SparkSession-86"><a href="#SparkSession-86"><span class="linenos"> 86</span></a> <span class="p">(</span>
|
</span><span id="SparkSession-86"><a href="#SparkSession-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||||
</span><span id="SparkSession-87"><a href="#SparkSession-87"><span class="linenos"> 87</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
</span><span id="SparkSession-87"><a href="#SparkSession-87"><span class="linenos"> 87</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
||||||
</span><span id="SparkSession-88"><a href="#SparkSession-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
</span><span id="SparkSession-88"><a href="#SparkSession-88"><span class="linenos"> 88</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession-89"><a href="#SparkSession-89"><span class="linenos"> 89</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
</span><span id="SparkSession-89"><a href="#SparkSession-89"><span class="linenos"> 89</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession-90"><a href="#SparkSession-90"><span class="linenos"> 90</span></a> <span class="p">)</span>
|
</span><span id="SparkSession-90"><a href="#SparkSession-90"><span class="linenos"> 90</span></a> <span class="p">]</span>
|
||||||
</span><span id="SparkSession-91"><a href="#SparkSession-91"><span class="linenos"> 91</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
</span><span id="SparkSession-91"><a href="#SparkSession-91"><span class="linenos"> 91</span></a>
|
||||||
</span><span id="SparkSession-92"><a href="#SparkSession-92"><span class="linenos"> 92</span></a> <span class="p">]</span>
|
</span><span id="SparkSession-92"><a href="#SparkSession-92"><span class="linenos"> 92</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
|
||||||
</span><span id="SparkSession-93"><a href="#SparkSession-93"><span class="linenos"> 93</span></a>
|
</span><span id="SparkSession-93"><a href="#SparkSession-93"><span class="linenos"> 93</span></a> <span class="s2">"expressions"</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-94"><a href="#SparkSession-94"><span class="linenos"> 94</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
|
</span><span id="SparkSession-94"><a href="#SparkSession-94"><span class="linenos"> 94</span></a> <span class="s2">"from"</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-95"><a href="#SparkSession-95"><span class="linenos"> 95</span></a> <span class="s2">"expressions"</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
|
</span><span id="SparkSession-95"><a href="#SparkSession-95"><span class="linenos"> 95</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-96"><a href="#SparkSession-96"><span class="linenos"> 96</span></a> <span class="s2">"from"</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
|
</span><span id="SparkSession-96"><a href="#SparkSession-96"><span class="linenos"> 96</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-97"><a href="#SparkSession-97"><span class="linenos"> 97</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
|
</span><span id="SparkSession-97"><a href="#SparkSession-97"><span class="linenos"> 97</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-98"><a href="#SparkSession-98"><span class="linenos"> 98</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
|
</span><span id="SparkSession-98"><a href="#SparkSession-98"><span class="linenos"> 98</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
|
||||||
</span><span id="SparkSession-99"><a href="#SparkSession-99"><span class="linenos"> 99</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
</span><span id="SparkSession-99"><a href="#SparkSession-99"><span class="linenos"> 99</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
|
||||||
</span><span id="SparkSession-100"><a href="#SparkSession-100"><span class="linenos">100</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
|
</span><span id="SparkSession-100"><a href="#SparkSession-100"><span class="linenos">100</span></a> <span class="p">),</span>
|
||||||
</span><span id="SparkSession-101"><a href="#SparkSession-101"><span class="linenos">101</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
|
</span><span id="SparkSession-101"><a href="#SparkSession-101"><span class="linenos">101</span></a> <span class="p">),</span>
|
||||||
</span><span id="SparkSession-102"><a href="#SparkSession-102"><span class="linenos">102</span></a> <span class="p">),</span>
|
</span><span id="SparkSession-102"><a href="#SparkSession-102"><span class="linenos">102</span></a> <span class="p">),</span>
|
||||||
</span><span id="SparkSession-103"><a href="#SparkSession-103"><span class="linenos">103</span></a> <span class="p">),</span>
|
</span><span id="SparkSession-103"><a href="#SparkSession-103"><span class="linenos">103</span></a> <span class="p">}</span>
|
||||||
</span><span id="SparkSession-104"><a href="#SparkSession-104"><span class="linenos">104</span></a> <span class="p">),</span>
|
</span><span id="SparkSession-104"><a href="#SparkSession-104"><span class="linenos">104</span></a>
|
||||||
</span><span id="SparkSession-105"><a href="#SparkSession-105"><span class="linenos">105</span></a> <span class="p">}</span>
|
</span><span id="SparkSession-105"><a href="#SparkSession-105"><span class="linenos">105</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-106"><a href="#SparkSession-106"><span class="linenos">106</span></a>
|
</span><span id="SparkSession-106"><a href="#SparkSession-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-107"><a href="#SparkSession-107"><span class="linenos">107</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
|
</span><span id="SparkSession-107"><a href="#SparkSession-107"><span class="linenos">107</span></a>
|
||||||
</span><span id="SparkSession-108"><a href="#SparkSession-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
|
</span><span id="SparkSession-108"><a href="#SparkSession-108"><span class="linenos">108</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">DataFrame</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-109"><a href="#SparkSession-109"><span class="linenos">109</span></a>
|
</span><span id="SparkSession-109"><a href="#SparkSession-109"><span class="linenos">109</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-110"><a href="#SparkSession-110"><span class="linenos">110</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">DataFrame</span><span class="p">:</span>
|
</span><span id="SparkSession-110"><a href="#SparkSession-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession-111"><a href="#SparkSession-111"><span class="linenos">111</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
</span><span id="SparkSession-111"><a href="#SparkSession-111"><span class="linenos">111</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-112"><a href="#SparkSession-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
</span><span id="SparkSession-112"><a href="#SparkSession-112"><span class="linenos">112</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession-113"><a href="#SparkSession-113"><span class="linenos">113</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
</span><span id="SparkSession-113"><a href="#SparkSession-113"><span class="linenos">113</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
|
||||||
</span><span id="SparkSession-114"><a href="#SparkSession-114"><span class="linenos">114</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
</span><span id="SparkSession-114"><a href="#SparkSession-114"><span class="linenos">114</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession-115"><a href="#SparkSession-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
|
</span><span id="SparkSession-115"><a href="#SparkSession-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession-116"><a href="#SparkSession-116"><span class="linenos">116</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
</span><span id="SparkSession-116"><a href="#SparkSession-116"><span class="linenos">116</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">))</span>
|
||||||
</span><span id="SparkSession-117"><a href="#SparkSession-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
|
</span><span id="SparkSession-117"><a href="#SparkSession-117"><span class="linenos">117</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-118"><a href="#SparkSession-118"><span class="linenos">118</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">))</span>
|
</span><span id="SparkSession-118"><a href="#SparkSession-118"><span class="linenos">118</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"expression"</span><span class="p">]</span>
|
||||||
</span><span id="SparkSession-119"><a href="#SparkSession-119"><span class="linenos">119</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
</span><span id="SparkSession-119"><a href="#SparkSession-119"><span class="linenos">119</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||||
</span><span id="SparkSession-120"><a href="#SparkSession-120"><span class="linenos">120</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"expression"</span><span class="p">]</span>
|
</span><span id="SparkSession-120"><a href="#SparkSession-120"><span class="linenos">120</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession-121"><a href="#SparkSession-121"><span class="linenos">121</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
</span><span id="SparkSession-121"><a href="#SparkSession-121"><span class="linenos">121</span></a> <span class="k">else</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-122"><a href="#SparkSession-122"><span class="linenos">122</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
</span><span id="SparkSession-122"><a href="#SparkSession-122"><span class="linenos">122</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-123"><a href="#SparkSession-123"><span class="linenos">123</span></a> <span class="k">else</span><span class="p">:</span>
|
</span><span id="SparkSession-123"><a href="#SparkSession-123"><span class="linenos">123</span></a> <span class="s2">"Unknown expression type provided in the SQL. Please create an issue with the SQL."</span>
|
||||||
</span><span id="SparkSession-124"><a href="#SparkSession-124"><span class="linenos">124</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
</span><span id="SparkSession-124"><a href="#SparkSession-124"><span class="linenos">124</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession-125"><a href="#SparkSession-125"><span class="linenos">125</span></a> <span class="s2">"Unknown expression type provided in the SQL. Please create an issue with the SQL."</span>
|
</span><span id="SparkSession-125"><a href="#SparkSession-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="n">df</span>
|
||||||
</span><span id="SparkSession-126"><a href="#SparkSession-126"><span class="linenos">126</span></a> <span class="p">)</span>
|
</span><span id="SparkSession-126"><a href="#SparkSession-126"><span class="linenos">126</span></a>
|
||||||
</span><span id="SparkSession-127"><a href="#SparkSession-127"><span class="linenos">127</span></a> <span class="k">return</span> <span class="n">df</span>
|
</span><span id="SparkSession-127"><a href="#SparkSession-127"><span class="linenos">127</span></a> <span class="nd">@property</span>
|
||||||
</span><span id="SparkSession-128"><a href="#SparkSession-128"><span class="linenos">128</span></a>
|
</span><span id="SparkSession-128"><a href="#SparkSession-128"><span class="linenos">128</span></a> <span class="k">def</span> <span class="nf">_auto_incrementing_name</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-129"><a href="#SparkSession-129"><span class="linenos">129</span></a> <span class="nd">@property</span>
|
</span><span id="SparkSession-129"><a href="#SparkSession-129"><span class="linenos">129</span></a> <span class="n">name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"a</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span><span class="si">}</span><span class="s2">"</span>
|
||||||
</span><span id="SparkSession-130"><a href="#SparkSession-130"><span class="linenos">130</span></a> <span class="k">def</span> <span class="nf">_auto_incrementing_name</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
</span><span id="SparkSession-130"><a href="#SparkSession-130"><span class="linenos">130</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||||
</span><span id="SparkSession-131"><a href="#SparkSession-131"><span class="linenos">131</span></a> <span class="n">name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"a</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span><span class="si">}</span><span class="s2">"</span>
|
</span><span id="SparkSession-131"><a href="#SparkSession-131"><span class="linenos">131</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||||
</span><span id="SparkSession-132"><a href="#SparkSession-132"><span class="linenos">132</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">incrementing_id</span> <span class="o">+=</span> <span class="mi">1</span>
|
</span><span id="SparkSession-132"><a href="#SparkSession-132"><span class="linenos">132</span></a>
|
||||||
</span><span id="SparkSession-133"><a href="#SparkSession-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">name</span>
|
</span><span id="SparkSession-133"><a href="#SparkSession-133"><span class="linenos">133</span></a> <span class="nd">@property</span>
|
||||||
</span><span id="SparkSession-134"><a href="#SparkSession-134"><span class="linenos">134</span></a>
|
</span><span id="SparkSession-134"><a href="#SparkSession-134"><span class="linenos">134</span></a> <span class="k">def</span> <span class="nf">_random_branch_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-135"><a href="#SparkSession-135"><span class="linenos">135</span></a> <span class="nd">@property</span>
|
</span><span id="SparkSession-135"><a href="#SparkSession-135"><span class="linenos">135</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
|
||||||
</span><span id="SparkSession-136"><a href="#SparkSession-136"><span class="linenos">136</span></a> <span class="k">def</span> <span class="nf">_random_branch_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
</span><span id="SparkSession-136"><a href="#SparkSession-136"><span class="linenos">136</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_branch_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-137"><a href="#SparkSession-137"><span class="linenos">137</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
|
</span><span id="SparkSession-137"><a href="#SparkSession-137"><span class="linenos">137</span></a> <span class="k">return</span> <span class="nb">id</span>
|
||||||
</span><span id="SparkSession-138"><a href="#SparkSession-138"><span class="linenos">138</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_branch_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
|
</span><span id="SparkSession-138"><a href="#SparkSession-138"><span class="linenos">138</span></a>
|
||||||
</span><span id="SparkSession-139"><a href="#SparkSession-139"><span class="linenos">139</span></a> <span class="k">return</span> <span class="nb">id</span>
|
</span><span id="SparkSession-139"><a href="#SparkSession-139"><span class="linenos">139</span></a> <span class="nd">@property</span>
|
||||||
</span><span id="SparkSession-140"><a href="#SparkSession-140"><span class="linenos">140</span></a>
|
</span><span id="SparkSession-140"><a href="#SparkSession-140"><span class="linenos">140</span></a> <span class="k">def</span> <span class="nf">_random_sequence_id</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession-141"><a href="#SparkSession-141"><span class="linenos">141</span></a> <span class="nd">@property</span>
|
</span><span id="SparkSession-141"><a href="#SparkSession-141"><span class="linenos">141</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
|
||||||
</span><span id="SparkSession-142"><a href="#SparkSession-142"><span class="linenos">142</span></a> <span class="k">def</span> <span class="nf">_random_sequence_id</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
</span><span id="SparkSession-142"><a href="#SparkSession-142"><span class="linenos">142</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_sequence_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-143"><a href="#SparkSession-143"><span class="linenos">143</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_random_id</span>
|
</span><span id="SparkSession-143"><a href="#SparkSession-143"><span class="linenos">143</span></a> <span class="k">return</span> <span class="nb">id</span>
|
||||||
</span><span id="SparkSession-144"><a href="#SparkSession-144"><span class="linenos">144</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_sequence_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
|
</span><span id="SparkSession-144"><a href="#SparkSession-144"><span class="linenos">144</span></a>
|
||||||
</span><span id="SparkSession-145"><a href="#SparkSession-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="nb">id</span>
|
</span><span id="SparkSession-145"><a href="#SparkSession-145"><span class="linenos">145</span></a> <span class="nd">@property</span>
|
||||||
</span><span id="SparkSession-146"><a href="#SparkSession-146"><span class="linenos">146</span></a>
|
</span><span id="SparkSession-146"><a href="#SparkSession-146"><span class="linenos">146</span></a> <span class="k">def</span> <span class="nf">_random_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-147"><a href="#SparkSession-147"><span class="linenos">147</span></a> <span class="nd">@property</span>
|
</span><span id="SparkSession-147"><a href="#SparkSession-147"><span class="linenos">147</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="s2">"r"</span> <span class="o">+</span> <span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span><span class="o">.</span><span class="n">hex</span>
|
||||||
</span><span id="SparkSession-148"><a href="#SparkSession-148"><span class="linenos">148</span></a> <span class="k">def</span> <span class="nf">_random_id</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
</span><span id="SparkSession-148"><a href="#SparkSession-148"><span class="linenos">148</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-149"><a href="#SparkSession-149"><span class="linenos">149</span></a> <span class="nb">id</span> <span class="o">=</span> <span class="s2">"r"</span> <span class="o">+</span> <span class="n">uuid</span><span class="o">.</span><span class="n">uuid4</span><span class="p">()</span><span class="o">.</span><span class="n">hex</span>
|
</span><span id="SparkSession-149"><a href="#SparkSession-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="nb">id</span>
|
||||||
</span><span id="SparkSession-150"><a href="#SparkSession-150"><span class="linenos">150</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">known_ids</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
|
</span><span id="SparkSession-150"><a href="#SparkSession-150"><span class="linenos">150</span></a>
|
||||||
</span><span id="SparkSession-151"><a href="#SparkSession-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="nb">id</span>
|
</span><span id="SparkSession-151"><a href="#SparkSession-151"><span class="linenos">151</span></a> <span class="nd">@property</span>
|
||||||
</span><span id="SparkSession-152"><a href="#SparkSession-152"><span class="linenos">152</span></a>
|
</span><span id="SparkSession-152"><a href="#SparkSession-152"><span class="linenos">152</span></a> <span class="k">def</span> <span class="nf">_join_hint_names</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||||
</span><span id="SparkSession-153"><a href="#SparkSession-153"><span class="linenos">153</span></a> <span class="nd">@property</span>
|
</span><span id="SparkSession-153"><a href="#SparkSession-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="p">{</span><span class="s2">"BROADCAST"</span><span class="p">,</span> <span class="s2">"MERGE"</span><span class="p">,</span> <span class="s2">"SHUFFLE_HASH"</span><span class="p">,</span> <span class="s2">"SHUFFLE_REPLICATE_NL"</span><span class="p">}</span>
|
||||||
</span><span id="SparkSession-154"><a href="#SparkSession-154"><span class="linenos">154</span></a> <span class="k">def</span> <span class="nf">_join_hint_names</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
</span><span id="SparkSession-154"><a href="#SparkSession-154"><span class="linenos">154</span></a>
|
||||||
</span><span id="SparkSession-155"><a href="#SparkSession-155"><span class="linenos">155</span></a> <span class="k">return</span> <span class="p">{</span><span class="s2">"BROADCAST"</span><span class="p">,</span> <span class="s2">"MERGE"</span><span class="p">,</span> <span class="s2">"SHUFFLE_HASH"</span><span class="p">,</span> <span class="s2">"SHUFFLE_REPLICATE_NL"</span><span class="p">}</span>
|
</span><span id="SparkSession-155"><a href="#SparkSession-155"><span class="linenos">155</span></a> <span class="k">def</span> <span class="nf">_add_alias_to_mapping</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">sequence_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession-156"><a href="#SparkSession-156"><span class="linenos">156</span></a>
|
</span><span id="SparkSession-156"><a href="#SparkSession-156"><span class="linenos">156</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name_to_sequence_id_mapping</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">sequence_id</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-157"><a href="#SparkSession-157"><span class="linenos">157</span></a> <span class="k">def</span> <span class="nf">_add_alias_to_mapping</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">sequence_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
|
</span><span id="SparkSession-157"><a href="#SparkSession-157"><span class="linenos">157</span></a>
|
||||||
</span><span id="SparkSession-158"><a href="#SparkSession-158"><span class="linenos">158</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name_to_sequence_id_mapping</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">sequence_id</span><span class="p">)</span>
|
</span><span id="SparkSession-158"><a href="#SparkSession-158"><span class="linenos">158</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-159"><a href="#SparkSession-159"><span class="linenos">159</span></a>
|
</span><span id="SparkSession-159"><a href="#SparkSession-159"><span class="linenos">159</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">"sqlframe.dialect"</span>
|
||||||
</span><span id="SparkSession-160"><a href="#SparkSession-160"><span class="linenos">160</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession-160"><a href="#SparkSession-160"><span class="linenos">160</span></a>
|
||||||
</span><span id="SparkSession-161"><a href="#SparkSession-161"><span class="linenos">161</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">"sqlframe.dialect"</span>
|
</span><span id="SparkSession-161"><a href="#SparkSession-161"><span class="linenos">161</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession-162"><a href="#SparkSession-162"><span class="linenos">162</span></a>
|
</span><span id="SparkSession-162"><a href="#SparkSession-162"><span class="linenos">162</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">"spark"</span>
|
||||||
</span><span id="SparkSession-163"><a href="#SparkSession-163"><span class="linenos">163</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
</span><span id="SparkSession-163"><a href="#SparkSession-163"><span class="linenos">163</span></a>
|
||||||
</span><span id="SparkSession-164"><a href="#SparkSession-164"><span class="linenos">164</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">"spark"</span>
|
</span><span id="SparkSession-164"><a href="#SparkSession-164"><span class="linenos">164</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-165"><a href="#SparkSession-165"><span class="linenos">165</span></a>
|
</span><span id="SparkSession-165"><a href="#SparkSession-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span><span id="SparkSession-166"><a href="#SparkSession-166"><span class="linenos">166</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession-166"><a href="#SparkSession-166"><span class="linenos">166</span></a>
|
||||||
</span><span id="SparkSession-167"><a href="#SparkSession-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession-167"><a href="#SparkSession-167"><span class="linenos">167</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession-168"><a href="#SparkSession-168"><span class="linenos">168</span></a>
|
</span><span id="SparkSession-168"><a href="#SparkSession-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span><span id="SparkSession-169"><a href="#SparkSession-169"><span class="linenos">169</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
</span><span id="SparkSession-169"><a href="#SparkSession-169"><span class="linenos">169</span></a>
|
||||||
</span><span id="SparkSession-170"><a href="#SparkSession-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession-170"><a href="#SparkSession-170"><span class="linenos">170</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession-171"><a href="#SparkSession-171"><span class="linenos">171</span></a>
|
</span><span id="SparkSession-171"><a href="#SparkSession-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-172"><a href="#SparkSession-172"><span class="linenos">172</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
|
</span><span id="SparkSession-172"><a href="#SparkSession-172"><span class="linenos">172</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-173"><a href="#SparkSession-173"><span class="linenos">173</span></a> <span class="bp">self</span><span class="p">,</span>
|
</span><span id="SparkSession-173"><a href="#SparkSession-173"><span class="linenos">173</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-174"><a href="#SparkSession-174"><span class="linenos">174</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession-174"><a href="#SparkSession-174"><span class="linenos">174</span></a> <span class="o">*</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-175"><a href="#SparkSession-175"><span class="linenos">175</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession-175"><a href="#SparkSession-175"><span class="linenos">175</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-176"><a href="#SparkSession-176"><span class="linenos">176</span></a> <span class="o">*</span><span class="p">,</span>
|
</span><span id="SparkSession-176"><a href="#SparkSession-176"><span class="linenos">176</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession-177"><a href="#SparkSession-177"><span class="linenos">177</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession-177"><a href="#SparkSession-177"><span class="linenos">177</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-178"><a href="#SparkSession-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
</span><span id="SparkSession-178"><a href="#SparkSession-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-179"><a href="#SparkSession-179"><span class="linenos">179</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession-179"><a href="#SparkSession-179"><span class="linenos">179</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
|
||||||
</span><span id="SparkSession-180"><a href="#SparkSession-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
|
</span><span id="SparkSession-180"><a href="#SparkSession-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-181"><a href="#SparkSession-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
|
</span><span id="SparkSession-181"><a href="#SparkSession-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
|
||||||
</span><span id="SparkSession-182"><a href="#SparkSession-182"><span class="linenos">182</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
|
</span><span id="SparkSession-182"><a href="#SparkSession-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span><span id="SparkSession-183"><a href="#SparkSession-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
|
</span><span id="SparkSession-183"><a href="#SparkSession-183"><span class="linenos">183</span></a>
|
||||||
</span><span id="SparkSession-184"><a href="#SparkSession-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession-184"><a href="#SparkSession-184"><span class="linenos">184</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-185"><a href="#SparkSession-185"><span class="linenos">185</span></a>
|
</span><span id="SparkSession-185"><a href="#SparkSession-185"><span class="linenos">185</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession-186"><a href="#SparkSession-186"><span class="linenos">186</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="p">:</span>
|
</span><span id="SparkSession-186"><a href="#SparkSession-186"><span class="linenos">186</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession-187"><a href="#SparkSession-187"><span class="linenos">187</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
</span><span id="SparkSession-187"><a href="#SparkSession-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">spark</span>
|
||||||
</span><span id="SparkSession-188"><a href="#SparkSession-188"><span class="linenos">188</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
</span><span id="SparkSession-188"><a href="#SparkSession-188"><span class="linenos">188</span></a>
|
||||||
</span><span id="SparkSession-189"><a href="#SparkSession-189"><span class="linenos">189</span></a> <span class="k">return</span> <span class="n">spark</span>
|
</span><span id="SparkSession-189"><a href="#SparkSession-189"><span class="linenos">189</span></a> <span class="nd">@classproperty</span>
|
||||||
</span><span id="SparkSession-190"><a href="#SparkSession-190"><span class="linenos">190</span></a>
|
</span><span id="SparkSession-190"><a href="#SparkSession-190"><span class="linenos">190</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-></span> <span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession-191"><a href="#SparkSession-191"><span class="linenos">191</span></a> <span class="nd">@classproperty</span>
|
</span><span id="SparkSession-191"><a href="#SparkSession-191"><span class="linenos">191</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession-192"><a href="#SparkSession-192"><span class="linenos">192</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-></span> <span class="n">Builder</span><span class="p">:</span>
|
|
||||||
</span><span id="SparkSession-193"><a href="#SparkSession-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
|
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -786,7 +784,7 @@
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763522736'</span><span class="o">></span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763522736'</span><span class="o">></span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595765140864'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981980193248'</span><span class="o">></span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981980193248'</span><span class="o">></span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978717392'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -820,41 +818,39 @@
|
||||||
</span><span id="SparkSession.createDataFrame-71"><a href="#SparkSession.createDataFrame-71"><span class="linenos"> 71</span></a> <span class="n">column_mapping</span> <span class="o">=</span> <span class="p">{</span><span class="sa">f</span><span class="s2">"_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">:</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)}</span>
|
</span><span id="SparkSession.createDataFrame-71"><a href="#SparkSession.createDataFrame-71"><span class="linenos"> 71</span></a> <span class="n">column_mapping</span> <span class="o">=</span> <span class="p">{</span><span class="sa">f</span><span class="s2">"_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">:</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)}</span>
|
||||||
</span><span id="SparkSession.createDataFrame-72"><a href="#SparkSession.createDataFrame-72"><span class="linenos"> 72</span></a>
|
</span><span id="SparkSession.createDataFrame-72"><a href="#SparkSession.createDataFrame-72"><span class="linenos"> 72</span></a>
|
||||||
</span><span id="SparkSession.createDataFrame-73"><a href="#SparkSession.createDataFrame-73"><span class="linenos"> 73</span></a> <span class="n">data_expressions</span> <span class="o">=</span> <span class="p">[</span>
|
</span><span id="SparkSession.createDataFrame-73"><a href="#SparkSession.createDataFrame-73"><span class="linenos"> 73</span></a> <span class="n">data_expressions</span> <span class="o">=</span> <span class="p">[</span>
|
||||||
</span><span id="SparkSession.createDataFrame-74"><a href="#SparkSession.createDataFrame-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Tuple</span><span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-74"><a href="#SparkSession.createDataFrame-74"><span class="linenos"> 74</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">tuple_</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.createDataFrame-75"><a href="#SparkSession.createDataFrame-75"><span class="linenos"> 75</span></a> <span class="n">expressions</span><span class="o">=</span><span class="nb">list</span><span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-75"><a href="#SparkSession.createDataFrame-75"><span class="linenos"> 75</span></a> <span class="o">*</span><span class="nb">map</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.createDataFrame-76"><a href="#SparkSession.createDataFrame-76"><span class="linenos"> 76</span></a> <span class="nb">map</span><span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-76"><a href="#SparkSession.createDataFrame-76"><span class="linenos"> 76</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.createDataFrame-77"><a href="#SparkSession.createDataFrame-77"><span class="linenos"> 77</span></a> <span class="k">lambda</span> <span class="n">x</span><span class="p">:</span> <span class="n">F</span><span class="o">.</span><span class="n">lit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
</span><span id="SparkSession.createDataFrame-77"><a href="#SparkSession.createDataFrame-77"><span class="linenos"> 77</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
|
||||||
</span><span id="SparkSession.createDataFrame-78"><a href="#SparkSession.createDataFrame-78"><span class="linenos"> 78</span></a> <span class="n">row</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">row</span><span class="p">,</span> <span class="nb">dict</span><span class="p">)</span> <span class="k">else</span> <span class="n">row</span><span class="o">.</span><span class="n">values</span><span class="p">(),</span>
|
</span><span id="SparkSession.createDataFrame-78"><a href="#SparkSession.createDataFrame-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession.createDataFrame-79"><a href="#SparkSession.createDataFrame-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
|
</span><span id="SparkSession.createDataFrame-79"><a href="#SparkSession.createDataFrame-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession.createDataFrame-80"><a href="#SparkSession.createDataFrame-80"><span class="linenos"> 80</span></a> <span class="p">)</span>
|
</span><span id="SparkSession.createDataFrame-80"><a href="#SparkSession.createDataFrame-80"><span class="linenos"> 80</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
|
||||||
</span><span id="SparkSession.createDataFrame-81"><a href="#SparkSession.createDataFrame-81"><span class="linenos"> 81</span></a> <span class="p">)</span>
|
</span><span id="SparkSession.createDataFrame-81"><a href="#SparkSession.createDataFrame-81"><span class="linenos"> 81</span></a> <span class="p">]</span>
|
||||||
</span><span id="SparkSession.createDataFrame-82"><a href="#SparkSession.createDataFrame-82"><span class="linenos"> 82</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">data</span>
|
</span><span id="SparkSession.createDataFrame-82"><a href="#SparkSession.createDataFrame-82"><span class="linenos"> 82</span></a>
|
||||||
</span><span id="SparkSession.createDataFrame-83"><a href="#SparkSession.createDataFrame-83"><span class="linenos"> 83</span></a> <span class="p">]</span>
|
</span><span id="SparkSession.createDataFrame-83"><a href="#SparkSession.createDataFrame-83"><span class="linenos"> 83</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
|
||||||
</span><span id="SparkSession.createDataFrame-84"><a href="#SparkSession.createDataFrame-84"><span class="linenos"> 84</span></a>
|
</span><span id="SparkSession.createDataFrame-84"><a href="#SparkSession.createDataFrame-84"><span class="linenos"> 84</span></a> <span class="p">(</span>
|
||||||
</span><span id="SparkSession.createDataFrame-85"><a href="#SparkSession.createDataFrame-85"><span class="linenos"> 85</span></a> <span class="n">sel_columns</span> <span class="o">=</span> <span class="p">[</span>
|
</span><span id="SparkSession.createDataFrame-85"><a href="#SparkSession.createDataFrame-85"><span class="linenos"> 85</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
||||||
</span><span id="SparkSession.createDataFrame-86"><a href="#SparkSession.createDataFrame-86"><span class="linenos"> 86</span></a> <span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-86"><a href="#SparkSession.createDataFrame-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||||
</span><span id="SparkSession.createDataFrame-87"><a href="#SparkSession.createDataFrame-87"><span class="linenos"> 87</span></a> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">data_type</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
</span><span id="SparkSession.createDataFrame-87"><a href="#SparkSession.createDataFrame-87"><span class="linenos"> 87</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
||||||
</span><span id="SparkSession.createDataFrame-88"><a href="#SparkSession.createDataFrame-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="n">data_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
</span><span id="SparkSession.createDataFrame-88"><a href="#SparkSession.createDataFrame-88"><span class="linenos"> 88</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession.createDataFrame-89"><a href="#SparkSession.createDataFrame-89"><span class="linenos"> 89</span></a> <span class="k">else</span> <span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">expression</span>
|
</span><span id="SparkSession.createDataFrame-89"><a href="#SparkSession.createDataFrame-89"><span class="linenos"> 89</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession.createDataFrame-90"><a href="#SparkSession.createDataFrame-90"><span class="linenos"> 90</span></a> <span class="p">)</span>
|
</span><span id="SparkSession.createDataFrame-90"><a href="#SparkSession.createDataFrame-90"><span class="linenos"> 90</span></a> <span class="p">]</span>
|
||||||
</span><span id="SparkSession.createDataFrame-91"><a href="#SparkSession.createDataFrame-91"><span class="linenos"> 91</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">data_type</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
</span><span id="SparkSession.createDataFrame-91"><a href="#SparkSession.createDataFrame-91"><span class="linenos"> 91</span></a>
|
||||||
</span><span id="SparkSession.createDataFrame-92"><a href="#SparkSession.createDataFrame-92"><span class="linenos"> 92</span></a> <span class="p">]</span>
|
</span><span id="SparkSession.createDataFrame-92"><a href="#SparkSession.createDataFrame-92"><span class="linenos"> 92</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
|
||||||
</span><span id="SparkSession.createDataFrame-93"><a href="#SparkSession.createDataFrame-93"><span class="linenos"> 93</span></a>
|
</span><span id="SparkSession.createDataFrame-93"><a href="#SparkSession.createDataFrame-93"><span class="linenos"> 93</span></a> <span class="s2">"expressions"</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.createDataFrame-94"><a href="#SparkSession.createDataFrame-94"><span class="linenos"> 94</span></a> <span class="n">select_kwargs</span> <span class="o">=</span> <span class="p">{</span>
|
</span><span id="SparkSession.createDataFrame-94"><a href="#SparkSession.createDataFrame-94"><span class="linenos"> 94</span></a> <span class="s2">"from"</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.createDataFrame-95"><a href="#SparkSession.createDataFrame-95"><span class="linenos"> 95</span></a> <span class="s2">"expressions"</span><span class="p">:</span> <span class="n">sel_columns</span><span class="p">,</span>
|
</span><span id="SparkSession.createDataFrame-95"><a href="#SparkSession.createDataFrame-95"><span class="linenos"> 95</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.createDataFrame-96"><a href="#SparkSession.createDataFrame-96"><span class="linenos"> 96</span></a> <span class="s2">"from"</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-96"><a href="#SparkSession.createDataFrame-96"><span class="linenos"> 96</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.createDataFrame-97"><a href="#SparkSession.createDataFrame-97"><span class="linenos"> 97</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-97"><a href="#SparkSession.createDataFrame-97"><span class="linenos"> 97</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.createDataFrame-98"><a href="#SparkSession.createDataFrame-98"><span class="linenos"> 98</span></a> <span class="n">expressions</span><span class="o">=</span><span class="n">data_expressions</span><span class="p">,</span>
|
</span><span id="SparkSession.createDataFrame-98"><a href="#SparkSession.createDataFrame-98"><span class="linenos"> 98</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
|
||||||
</span><span id="SparkSession.createDataFrame-99"><a href="#SparkSession.createDataFrame-99"><span class="linenos"> 99</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
</span><span id="SparkSession.createDataFrame-99"><a href="#SparkSession.createDataFrame-99"><span class="linenos"> 99</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
|
||||||
</span><span id="SparkSession.createDataFrame-100"><a href="#SparkSession.createDataFrame-100"><span class="linenos">100</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_auto_incrementing_name</span><span class="p">),</span>
|
</span><span id="SparkSession.createDataFrame-100"><a href="#SparkSession.createDataFrame-100"><span class="linenos">100</span></a> <span class="p">),</span>
|
||||||
</span><span id="SparkSession.createDataFrame-101"><a href="#SparkSession.createDataFrame-101"><span class="linenos">101</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">col_name</span> <span class="ow">in</span> <span class="n">column_mapping</span><span class="p">],</span>
|
</span><span id="SparkSession.createDataFrame-101"><a href="#SparkSession.createDataFrame-101"><span class="linenos">101</span></a> <span class="p">),</span>
|
||||||
</span><span id="SparkSession.createDataFrame-102"><a href="#SparkSession.createDataFrame-102"><span class="linenos">102</span></a> <span class="p">),</span>
|
</span><span id="SparkSession.createDataFrame-102"><a href="#SparkSession.createDataFrame-102"><span class="linenos">102</span></a> <span class="p">),</span>
|
||||||
</span><span id="SparkSession.createDataFrame-103"><a href="#SparkSession.createDataFrame-103"><span class="linenos">103</span></a> <span class="p">),</span>
|
</span><span id="SparkSession.createDataFrame-103"><a href="#SparkSession.createDataFrame-103"><span class="linenos">103</span></a> <span class="p">}</span>
|
||||||
</span><span id="SparkSession.createDataFrame-104"><a href="#SparkSession.createDataFrame-104"><span class="linenos">104</span></a> <span class="p">),</span>
|
</span><span id="SparkSession.createDataFrame-104"><a href="#SparkSession.createDataFrame-104"><span class="linenos">104</span></a>
|
||||||
</span><span id="SparkSession.createDataFrame-105"><a href="#SparkSession.createDataFrame-105"><span class="linenos">105</span></a> <span class="p">}</span>
|
</span><span id="SparkSession.createDataFrame-105"><a href="#SparkSession.createDataFrame-105"><span class="linenos">105</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.createDataFrame-106"><a href="#SparkSession.createDataFrame-106"><span class="linenos">106</span></a>
|
</span><span id="SparkSession.createDataFrame-106"><a href="#SparkSession.createDataFrame-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.createDataFrame-107"><a href="#SparkSession.createDataFrame-107"><span class="linenos">107</span></a> <span class="n">sel_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">(</span><span class="o">**</span><span class="n">select_kwargs</span><span class="p">)</span>
|
|
||||||
</span><span id="SparkSession.createDataFrame-108"><a href="#SparkSession.createDataFrame-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sel_expression</span><span class="p">)</span>
|
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -872,24 +868,24 @@
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#SparkSession.sql"></a>
|
<a class="headerlink" href="#SparkSession.sql"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.sql-110"><a href="#SparkSession.sql-110"><span class="linenos">110</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">DataFrame</span><span class="p">:</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.sql-108"><a href="#SparkSession.sql-108"><span class="linenos">108</span></a> <span class="k">def</span> <span class="nf">sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">sqlQuery</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">DataFrame</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.sql-111"><a href="#SparkSession.sql-111"><span class="linenos">111</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
</span><span id="SparkSession.sql-109"><a href="#SparkSession.sql-109"><span class="linenos">109</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">parse_one</span><span class="p">(</span><span class="n">sqlQuery</span><span class="p">,</span> <span class="n">read</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.sql-112"><a href="#SparkSession.sql-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
</span><span id="SparkSession.sql-110"><a href="#SparkSession.sql-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession.sql-113"><a href="#SparkSession.sql-113"><span class="linenos">113</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
</span><span id="SparkSession.sql-111"><a href="#SparkSession.sql-111"><span class="linenos">111</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.sql-114"><a href="#SparkSession.sql-114"><span class="linenos">114</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
</span><span id="SparkSession.sql-112"><a href="#SparkSession.sql-112"><span class="linenos">112</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession.sql-115"><a href="#SparkSession.sql-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
|
</span><span id="SparkSession.sql-113"><a href="#SparkSession.sql-113"><span class="linenos">113</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Create</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">)):</span>
|
||||||
</span><span id="SparkSession.sql-116"><a href="#SparkSession.sql-116"><span class="linenos">116</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
</span><span id="SparkSession.sql-114"><a href="#SparkSession.sql-114"><span class="linenos">114</span></a> <span class="n">select_expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession.sql-117"><a href="#SparkSession.sql-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
|
</span><span id="SparkSession.sql-115"><a href="#SparkSession.sql-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Insert</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession.sql-118"><a href="#SparkSession.sql-118"><span class="linenos">118</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">))</span>
|
</span><span id="SparkSession.sql-116"><a href="#SparkSession.sql-116"><span class="linenos">116</span></a> <span class="n">select_expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">))</span>
|
||||||
</span><span id="SparkSession.sql-119"><a href="#SparkSession.sql-119"><span class="linenos">119</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
</span><span id="SparkSession.sql-117"><a href="#SparkSession.sql-117"><span class="linenos">117</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"with"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.sql-120"><a href="#SparkSession.sql-120"><span class="linenos">120</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"expression"</span><span class="p">]</span>
|
</span><span id="SparkSession.sql-118"><a href="#SparkSession.sql-118"><span class="linenos">118</span></a> <span class="k">del</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"expression"</span><span class="p">]</span>
|
||||||
</span><span id="SparkSession.sql-121"><a href="#SparkSession.sql-121"><span class="linenos">121</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
</span><span id="SparkSession.sql-119"><a href="#SparkSession.sql-119"><span class="linenos">119</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">select_expression</span><span class="p">,</span> <span class="n">output_expression_container</span><span class="o">=</span><span class="n">expression</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||||
</span><span id="SparkSession.sql-122"><a href="#SparkSession.sql-122"><span class="linenos">122</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
</span><span id="SparkSession.sql-120"><a href="#SparkSession.sql-120"><span class="linenos">120</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">_convert_leaf_to_cte</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession.sql-123"><a href="#SparkSession.sql-123"><span class="linenos">123</span></a> <span class="k">else</span><span class="p">:</span>
|
</span><span id="SparkSession.sql-121"><a href="#SparkSession.sql-121"><span class="linenos">121</span></a> <span class="k">else</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.sql-124"><a href="#SparkSession.sql-124"><span class="linenos">124</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
</span><span id="SparkSession.sql-122"><a href="#SparkSession.sql-122"><span class="linenos">122</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.sql-125"><a href="#SparkSession.sql-125"><span class="linenos">125</span></a> <span class="s2">"Unknown expression type provided in the SQL. Please create an issue with the SQL."</span>
|
</span><span id="SparkSession.sql-123"><a href="#SparkSession.sql-123"><span class="linenos">123</span></a> <span class="s2">"Unknown expression type provided in the SQL. Please create an issue with the SQL."</span>
|
||||||
</span><span id="SparkSession.sql-126"><a href="#SparkSession.sql-126"><span class="linenos">126</span></a> <span class="p">)</span>
|
</span><span id="SparkSession.sql-124"><a href="#SparkSession.sql-124"><span class="linenos">124</span></a> <span class="p">)</span>
|
||||||
</span><span id="SparkSession.sql-127"><a href="#SparkSession.sql-127"><span class="linenos">127</span></a> <span class="k">return</span> <span class="n">df</span>
|
</span><span id="SparkSession.sql-125"><a href="#SparkSession.sql-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="n">df</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -905,9 +901,9 @@
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#SparkSession.builder"></a>
|
<a class="headerlink" href="#SparkSession.builder"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.builder-191"><a href="#SparkSession.builder-191"><span class="linenos">191</span></a> <span class="nd">@classproperty</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.builder-189"><a href="#SparkSession.builder-189"><span class="linenos">189</span></a> <span class="nd">@classproperty</span>
|
||||||
</span><span id="SparkSession.builder-192"><a href="#SparkSession.builder-192"><span class="linenos">192</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-></span> <span class="n">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession.builder-190"><a href="#SparkSession.builder-190"><span class="linenos">190</span></a> <span class="k">def</span> <span class="nf">builder</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span> <span class="o">-></span> <span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.builder-193"><a href="#SparkSession.builder-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
|
</span><span id="SparkSession.builder-191"><a href="#SparkSession.builder-191"><span class="linenos">191</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="o">.</span><span class="n">Builder</span><span class="p">()</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -926,36 +922,36 @@
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#SparkSession.Builder"></a>
|
<a class="headerlink" href="#SparkSession.Builder"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder-160"><a href="#SparkSession.Builder-160"><span class="linenos">160</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder-158"><a href="#SparkSession.Builder-158"><span class="linenos">158</span></a> <span class="k">class</span> <span class="nc">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder-161"><a href="#SparkSession.Builder-161"><span class="linenos">161</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">"sqlframe.dialect"</span>
|
</span><span id="SparkSession.Builder-159"><a href="#SparkSession.Builder-159"><span class="linenos">159</span></a> <span class="n">SQLFRAME_DIALECT_KEY</span> <span class="o">=</span> <span class="s2">"sqlframe.dialect"</span>
|
||||||
</span><span id="SparkSession.Builder-162"><a href="#SparkSession.Builder-162"><span class="linenos">162</span></a>
|
</span><span id="SparkSession.Builder-160"><a href="#SparkSession.Builder-160"><span class="linenos">160</span></a>
|
||||||
</span><span id="SparkSession.Builder-163"><a href="#SparkSession.Builder-163"><span class="linenos">163</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
</span><span id="SparkSession.Builder-161"><a href="#SparkSession.Builder-161"><span class="linenos">161</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession.Builder-164"><a href="#SparkSession.Builder-164"><span class="linenos">164</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">"spark"</span>
|
</span><span id="SparkSession.Builder-162"><a href="#SparkSession.Builder-162"><span class="linenos">162</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="s2">"spark"</span>
|
||||||
</span><span id="SparkSession.Builder-165"><a href="#SparkSession.Builder-165"><span class="linenos">165</span></a>
|
</span><span id="SparkSession.Builder-163"><a href="#SparkSession.Builder-163"><span class="linenos">163</span></a>
|
||||||
</span><span id="SparkSession.Builder-166"><a href="#SparkSession.Builder-166"><span class="linenos">166</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder-164"><a href="#SparkSession.Builder-164"><span class="linenos">164</span></a> <span class="k">def</span> <span class="fm">__getattr__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder-167"><a href="#SparkSession.Builder-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession.Builder-165"><a href="#SparkSession.Builder-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span><span id="SparkSession.Builder-168"><a href="#SparkSession.Builder-168"><span class="linenos">168</span></a>
|
</span><span id="SparkSession.Builder-166"><a href="#SparkSession.Builder-166"><span class="linenos">166</span></a>
|
||||||
</span><span id="SparkSession.Builder-169"><a href="#SparkSession.Builder-169"><span class="linenos">169</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
</span><span id="SparkSession.Builder-167"><a href="#SparkSession.Builder-167"><span class="linenos">167</span></a> <span class="k">def</span> <span class="fm">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
|
||||||
</span><span id="SparkSession.Builder-170"><a href="#SparkSession.Builder-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession.Builder-168"><a href="#SparkSession.Builder-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span><span id="SparkSession.Builder-171"><a href="#SparkSession.Builder-171"><span class="linenos">171</span></a>
|
</span><span id="SparkSession.Builder-169"><a href="#SparkSession.Builder-169"><span class="linenos">169</span></a>
|
||||||
</span><span id="SparkSession.Builder-172"><a href="#SparkSession.Builder-172"><span class="linenos">172</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
|
</span><span id="SparkSession.Builder-170"><a href="#SparkSession.Builder-170"><span class="linenos">170</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.Builder-173"><a href="#SparkSession.Builder-173"><span class="linenos">173</span></a> <span class="bp">self</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder-171"><a href="#SparkSession.Builder-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder-174"><a href="#SparkSession.Builder-174"><span class="linenos">174</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder-172"><a href="#SparkSession.Builder-172"><span class="linenos">172</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder-175"><a href="#SparkSession.Builder-175"><span class="linenos">175</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder-173"><a href="#SparkSession.Builder-173"><span class="linenos">173</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder-176"><a href="#SparkSession.Builder-176"><span class="linenos">176</span></a> <span class="o">*</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder-174"><a href="#SparkSession.Builder-174"><span class="linenos">174</span></a> <span class="o">*</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder-177"><a href="#SparkSession.Builder-177"><span class="linenos">177</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder-175"><a href="#SparkSession.Builder-175"><span class="linenos">175</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder-178"><a href="#SparkSession.Builder-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder-176"><a href="#SparkSession.Builder-176"><span class="linenos">176</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder-179"><a href="#SparkSession.Builder-179"><span class="linenos">179</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder-177"><a href="#SparkSession.Builder-177"><span class="linenos">177</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder-180"><a href="#SparkSession.Builder-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder-178"><a href="#SparkSession.Builder-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder-181"><a href="#SparkSession.Builder-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
|
</span><span id="SparkSession.Builder-179"><a href="#SparkSession.Builder-179"><span class="linenos">179</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
|
||||||
</span><span id="SparkSession.Builder-182"><a href="#SparkSession.Builder-182"><span class="linenos">182</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder-180"><a href="#SparkSession.Builder-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder-183"><a href="#SparkSession.Builder-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
|
</span><span id="SparkSession.Builder-181"><a href="#SparkSession.Builder-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
|
||||||
</span><span id="SparkSession.Builder-184"><a href="#SparkSession.Builder-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession.Builder-182"><a href="#SparkSession.Builder-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span><span id="SparkSession.Builder-185"><a href="#SparkSession.Builder-185"><span class="linenos">185</span></a>
|
</span><span id="SparkSession.Builder-183"><a href="#SparkSession.Builder-183"><span class="linenos">183</span></a>
|
||||||
</span><span id="SparkSession.Builder-186"><a href="#SparkSession.Builder-186"><span class="linenos">186</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder-184"><a href="#SparkSession.Builder-184"><span class="linenos">184</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder-187"><a href="#SparkSession.Builder-187"><span class="linenos">187</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
</span><span id="SparkSession.Builder-185"><a href="#SparkSession.Builder-185"><span class="linenos">185</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession.Builder-188"><a href="#SparkSession.Builder-188"><span class="linenos">188</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
</span><span id="SparkSession.Builder-186"><a href="#SparkSession.Builder-186"><span class="linenos">186</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.Builder-189"><a href="#SparkSession.Builder-189"><span class="linenos">189</span></a> <span class="k">return</span> <span class="n">spark</span>
|
</span><span id="SparkSession.Builder-187"><a href="#SparkSession.Builder-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">spark</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -995,19 +991,19 @@
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#SparkSession.Builder.config"></a>
|
<a class="headerlink" href="#SparkSession.Builder.config"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.config-172"><a href="#SparkSession.Builder.config-172"><span class="linenos">172</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.config-170"><a href="#SparkSession.Builder.config-170"><span class="linenos">170</span></a> <span class="k">def</span> <span class="nf">config</span><span class="p">(</span>
|
||||||
</span><span id="SparkSession.Builder.config-173"><a href="#SparkSession.Builder.config-173"><span class="linenos">173</span></a> <span class="bp">self</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder.config-171"><a href="#SparkSession.Builder.config-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder.config-174"><a href="#SparkSession.Builder.config-174"><span class="linenos">174</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder.config-172"><a href="#SparkSession.Builder.config-172"><span class="linenos">172</span></a> <span class="n">key</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder.config-175"><a href="#SparkSession.Builder.config-175"><span class="linenos">175</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder.config-173"><a href="#SparkSession.Builder.config-173"><span class="linenos">173</span></a> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder.config-176"><a href="#SparkSession.Builder.config-176"><span class="linenos">176</span></a> <span class="o">*</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder.config-174"><a href="#SparkSession.Builder.config-174"><span class="linenos">174</span></a> <span class="o">*</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder.config-177"><a href="#SparkSession.Builder.config-177"><span class="linenos">177</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder.config-175"><a href="#SparkSession.Builder.config-175"><span class="linenos">175</span></a> <span class="nb">map</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder.config-178"><a href="#SparkSession.Builder.config-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
</span><span id="SparkSession.Builder.config-176"><a href="#SparkSession.Builder.config-176"><span class="linenos">176</span></a> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||||
</span><span id="SparkSession.Builder.config-179"><a href="#SparkSession.Builder.config-179"><span class="linenos">179</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder.config-177"><a href="#SparkSession.Builder.config-177"><span class="linenos">177</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">Builder</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder.config-180"><a href="#SparkSession.Builder.config-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder.config-178"><a href="#SparkSession.Builder.config-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">==</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder.config-181"><a href="#SparkSession.Builder.config-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
|
</span><span id="SparkSession.Builder.config-179"><a href="#SparkSession.Builder.config-179"><span class="linenos">179</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">value</span>
|
||||||
</span><span id="SparkSession.Builder.config-182"><a href="#SparkSession.Builder.config-182"><span class="linenos">182</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
|
</span><span id="SparkSession.Builder.config-180"><a href="#SparkSession.Builder.config-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">map</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span> <span class="ow">in</span> <span class="nb">map</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder.config-183"><a href="#SparkSession.Builder.config-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
|
</span><span id="SparkSession.Builder.config-181"><a href="#SparkSession.Builder.config-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="nb">map</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">SQLFRAME_DIALECT_KEY</span><span class="p">]</span>
|
||||||
</span><span id="SparkSession.Builder.config-184"><a href="#SparkSession.Builder.config-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="bp">self</span>
|
</span><span id="SparkSession.Builder.config-182"><a href="#SparkSession.Builder.config-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1025,10 +1021,10 @@
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#SparkSession.Builder.getOrCreate"></a>
|
<a class="headerlink" href="#SparkSession.Builder.getOrCreate"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.getOrCreate-186"><a href="#SparkSession.Builder.getOrCreate-186"><span class="linenos">186</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="p">:</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="SparkSession.Builder.getOrCreate-184"><a href="#SparkSession.Builder.getOrCreate-184"><span class="linenos">184</span></a> <span class="k">def</span> <span class="nf">getOrCreate</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">SparkSession</span><span class="p">:</span>
|
||||||
</span><span id="SparkSession.Builder.getOrCreate-187"><a href="#SparkSession.Builder.getOrCreate-187"><span class="linenos">187</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
</span><span id="SparkSession.Builder.getOrCreate-185"><a href="#SparkSession.Builder.getOrCreate-185"><span class="linenos">185</span></a> <span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
||||||
</span><span id="SparkSession.Builder.getOrCreate-188"><a href="#SparkSession.Builder.getOrCreate-188"><span class="linenos">188</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
</span><span id="SparkSession.Builder.getOrCreate-186"><a href="#SparkSession.Builder.getOrCreate-186"><span class="linenos">186</span></a> <span class="n">spark</span><span class="o">.</span><span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||||
</span><span id="SparkSession.Builder.getOrCreate-189"><a href="#SparkSession.Builder.getOrCreate-189"><span class="linenos">189</span></a> <span class="k">return</span> <span class="n">spark</span>
|
</span><span id="SparkSession.Builder.getOrCreate-187"><a href="#SparkSession.Builder.getOrCreate-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">spark</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1844,7 +1840,7 @@
|
||||||
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595768209808'</span><span class="o">></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o"><</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">></span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595768526640'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
|
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981983176912'</span><span class="o">></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o"><</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">></span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981983444528'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
|
||||||
|
|
||||||
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -2092,7 +2088,7 @@
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763912496'</span><span class="o">></span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
|
<span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981979212416'</span><span class="o">></span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -2841,7 +2837,7 @@ is unlikely to come up.</p>
|
||||||
<div class="decorator">@operation(Operation.FROM)</div>
|
<div class="decorator">@operation(Operation.FROM)</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595764027520'</span><span class="o">></span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981977951568'</span><span class="o">></span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -2910,7 +2906,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="decorator">@operation(Operation.FROM)</div>
|
<div class="decorator">@operation(Operation.FROM)</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595762701712'</span><span class="o">></span><span class="p">],</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595762701712'</span><span class="o">></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981977374288'</span><span class="o">></span><span class="p">],</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981977374288'</span><span class="o">></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3115,7 +3111,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="decorator">@operation(Operation.NO_OP)</div>
|
<div class="decorator">@operation(Operation.NO_OP)</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595762701712'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595762701712'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981977374288'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981977374288'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3835,7 +3831,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
|
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3879,7 +3875,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="decorator">@classmethod</div>
|
<div class="decorator">@classmethod</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3900,7 +3896,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="decorator">@classmethod</div>
|
<div class="decorator">@classmethod</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span>
|
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3921,7 +3917,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="decorator">@classmethod</div>
|
<div class="decorator">@classmethod</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3948,7 +3944,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="decorator">@classmethod</div>
|
<div class="decorator">@classmethod</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -3987,7 +3983,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -4008,7 +4004,7 @@ and check if it matches the type of the value provided. If not then make it null
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -4602,7 +4598,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
|
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -4623,7 +4619,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595766433424'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981981343056'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -4658,7 +4654,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595762334880'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981977905056'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -4903,7 +4899,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="decorator">@classmethod</div>
|
<div class="decorator">@classmethod</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -4924,7 +4920,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="decorator">@classmethod</div>
|
<div class="decorator">@classmethod</div>
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -5166,7 +5162,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
@ -5193,7 +5189,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
||||||
<div class="attr function">
|
<div class="attr function">
|
||||||
|
|
||||||
<span class="def">def</span>
|
<span class="def">def</span>
|
||||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140595763234272'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139981978118480'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
|
||||||
|
|
||||||
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>
|
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>
|
||||||
|
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -1808,7 +1808,7 @@ belong to some totally-ordered set.</p>
|
||||||
<section id="DATE_UNITS">
|
<section id="DATE_UNITS">
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">DATE_UNITS</span> =
|
<span class="name">DATE_UNITS</span> =
|
||||||
<span class="default_value">{'year_month', 'quarter', 'month', 'year', 'week', 'day'}</span>
|
<span class="default_value">{'quarter', 'month', 'day', 'year', 'week', 'year_month'}</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
|
|
@ -578,7 +578,7 @@
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">ALL_JSON_PATH_PARTS</span> =
|
<span class="name">ALL_JSON_PATH_PARTS</span> =
|
||||||
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||||
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{<class '<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>'>, <class '<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>'>, <class '<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>'>, <class '<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>'>, <class '<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>'>, <class '<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>'>, <class '<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>'>, <class '<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>'>, <class '<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>'>, <class '<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>'>}</span>
|
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{<class '<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>'>, <class '<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>'>, <class '<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>'>, <class '<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>'>, <class '<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>'>, <class '<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>'>, <class '<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>'>, <class '<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>'>, <class '<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>'>, <class '<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>'>}</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -117,137 +117,141 @@
|
||||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
||||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
|
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
|
||||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">return</span> <span class="n">node</span>
|
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
|
||||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
|
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
|
||||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a>
|
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
|
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
||||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
|
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
|
||||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
|
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
|
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
|
||||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
|
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a>
|
||||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
|
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
|
||||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
|
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
|
||||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
|
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
|
||||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
|
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
|
||||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
|
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
|
||||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
|
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
|
||||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="p">)</span>
|
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
|
||||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
|
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
|
||||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a>
|
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
|
||||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
|
||||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
|
||||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a><span class="p">)</span>
|
||||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
|
||||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
|
||||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
||||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="p">):</span>
|
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
||||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
||||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
||||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="p">):</span>
|
||||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a>
|
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">return</span> <span class="n">node</span>
|
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
||||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
|
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a>
|
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
||||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">if</span> <span class="p">(</span>
|
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
|
||||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span>
|
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
|
||||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
|
||||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="p">):</span>
|
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
||||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">return</span> <span class="n">expression</span>
|
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
|
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="p">):</span>
|
||||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a>
|
||||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
|
||||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
||||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
||||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
||||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
||||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">):</span>
|
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
||||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="p">):</span>
|
||||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">return</span> <span class="n">expression</span>
|
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
||||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
|
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
|
||||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a>
|
||||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
|
||||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
|
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="n">expression</span>
|
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
||||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
|
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
||||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
|
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
|
||||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
|
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
|
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
|
||||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
|
||||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="p">(</span>
|
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
|
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
|
||||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
|
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
|
||||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
|
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="ow">not</span> <span class="ow">in</span> <span class="p">(</span>
|
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
|
||||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">,</span>
|
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
|
||||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">INTERVAL</span><span class="p">,</span>
|
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
|
||||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="p">)</span>
|
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">):</span>
|
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="ow">not</span> <span class="ow">in</span> <span class="p">(</span>
|
||||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">,</span>
|
||||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
|
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">INTERVAL</span><span class="p">,</span>
|
||||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
|
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="p">)</span>
|
||||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="p">):</span>
|
||||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">arg</span>
|
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a>
|
||||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
|
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
|
||||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
|
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
|
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
|
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">return</span> <span class="n">arg</span>
|
||||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
|
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
|
||||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
|
||||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
|
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
|
||||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
|
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
|
||||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
|
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
|
||||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
|
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
||||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
|
||||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
|
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
|
||||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
|
||||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
|
||||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
|
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="n">arg</span>
|
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
|
||||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
|
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
|
||||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
|
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
|
||||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
|
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">arg</span>
|
||||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
|
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
|
||||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
|
||||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
|
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a>
|
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
|
||||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
|
||||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
|
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
|
||||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
||||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
|
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
|
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
|
||||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a><span class="c1"># presto has a boolean type whereas tsql doesn't (people use bits)</span>
|
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
|
||||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
|
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a>
|
||||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
|
||||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
|
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
|
||||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
|
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a><span class="c1"># presto has a boolean type whereas tsql doesn't (people use bits)</span>
|
||||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
|
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
|
||||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
|
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
|
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
|
||||||
|
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
|
||||||
|
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
|
||||||
|
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
|
||||||
|
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -335,8 +339,13 @@ conversions rely on type inference.</p>
|
||||||
</span><span id="replace_date_funcs-39"><a href="#replace_date_funcs-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
</span><span id="replace_date_funcs-39"><a href="#replace_date_funcs-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"zone"</span><span class="p">):</span>
|
||||||
</span><span id="replace_date_funcs-40"><a href="#replace_date_funcs-40"><span class="linenos">40</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
</span><span id="replace_date_funcs-40"><a href="#replace_date_funcs-40"><span class="linenos">40</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
|
||||||
</span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
|
</span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
|
||||||
</span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
</span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
|
||||||
</span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="k">return</span> <span class="n">node</span>
|
</span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
|
||||||
|
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a>
|
||||||
|
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||||
|
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
|
||||||
|
</span><span id="replace_date_funcs-47"><a href="#replace_date_funcs-47"><span class="linenos">47</span></a>
|
||||||
|
</span><span id="replace_date_funcs-48"><a href="#replace_date_funcs-48"><span class="linenos">48</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -367,21 +376,21 @@ conversions rely on type inference.</p>
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#coerce_type"></a>
|
<a class="headerlink" href="#coerce_type"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-60"><a href="#coerce_type-60"><span class="linenos">60</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-65"><a href="#coerce_type-65"><span class="linenos">65</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="coerce_type-61"><a href="#coerce_type-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
</span><span id="coerce_type-66"><a href="#coerce_type-66"><span class="linenos">66</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
|
||||||
</span><span id="coerce_type-62"><a href="#coerce_type-62"><span class="linenos">62</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
</span><span id="coerce_type-67"><a href="#coerce_type-67"><span class="linenos">67</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||||
</span><span id="coerce_type-63"><a href="#coerce_type-63"><span class="linenos">63</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
</span><span id="coerce_type-68"><a href="#coerce_type-68"><span class="linenos">68</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
|
||||||
</span><span id="coerce_type-64"><a href="#coerce_type-64"><span class="linenos">64</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
</span><span id="coerce_type-69"><a href="#coerce_type-69"><span class="linenos">69</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"low"</span><span class="p">])</span>
|
||||||
</span><span id="coerce_type-65"><a href="#coerce_type-65"><span class="linenos">65</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
</span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
|
||||||
</span><span id="coerce_type-66"><a href="#coerce_type-66"><span class="linenos">66</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||||
</span><span id="coerce_type-67"><a href="#coerce_type-67"><span class="linenos">67</span></a> <span class="p">):</span>
|
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="p">):</span>
|
||||||
</span><span id="coerce_type-68"><a href="#coerce_type-68"><span class="linenos">68</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
|
||||||
</span><span id="coerce_type-69"><a href="#coerce_type-69"><span class="linenos">69</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
|
||||||
</span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
</span><span id="coerce_type-75"><a href="#coerce_type-75"><span class="linenos">75</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
|
||||||
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
</span><span id="coerce_type-76"><a href="#coerce_type-76"><span class="linenos">76</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
|
||||||
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
</span><span id="coerce_type-77"><a href="#coerce_type-77"><span class="linenos">77</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||||
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a>
|
</span><span id="coerce_type-78"><a href="#coerce_type-78"><span class="linenos">78</span></a>
|
||||||
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="k">return</span> <span class="n">node</span>
|
</span><span id="coerce_type-79"><a href="#coerce_type-79"><span class="linenos">79</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -399,15 +408,14 @@ conversions rely on type inference.</p>
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#remove_redundant_casts"></a>
|
<a class="headerlink" href="#remove_redundant_casts"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-77"><a href="#remove_redundant_casts-77"><span class="linenos">77</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-82"><a href="#remove_redundant_casts-82"><span class="linenos">82</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="remove_redundant_casts-78"><a href="#remove_redundant_casts-78"><span class="linenos">78</span></a> <span class="k">if</span> <span class="p">(</span>
|
</span><span id="remove_redundant_casts-83"><a href="#remove_redundant_casts-83"><span class="linenos">83</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||||
</span><span id="remove_redundant_casts-79"><a href="#remove_redundant_casts-79"><span class="linenos">79</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
</span><span id="remove_redundant_casts-84"><a href="#remove_redundant_casts-84"><span class="linenos">84</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
|
||||||
</span><span id="remove_redundant_casts-80"><a href="#remove_redundant_casts-80"><span class="linenos">80</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span>
|
</span><span id="remove_redundant_casts-85"><a href="#remove_redundant_casts-85"><span class="linenos">85</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
||||||
</span><span id="remove_redundant_casts-81"><a href="#remove_redundant_casts-81"><span class="linenos">81</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
|
</span><span id="remove_redundant_casts-86"><a href="#remove_redundant_casts-86"><span class="linenos">86</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="remove_redundant_casts-82"><a href="#remove_redundant_casts-82"><span class="linenos">82</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="remove_redundant_casts-87"><a href="#remove_redundant_casts-87"><span class="linenos">87</span></a> <span class="p">):</span>
|
||||||
</span><span id="remove_redundant_casts-83"><a href="#remove_redundant_casts-83"><span class="linenos">83</span></a> <span class="p">):</span>
|
</span><span id="remove_redundant_casts-88"><a href="#remove_redundant_casts-88"><span class="linenos">88</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="remove_redundant_casts-84"><a href="#remove_redundant_casts-84"><span class="linenos">84</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="remove_redundant_casts-89"><a href="#remove_redundant_casts-89"><span class="linenos">89</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||||
</span><span id="remove_redundant_casts-85"><a href="#remove_redundant_casts-85"><span class="linenos">85</span></a> <span class="k">return</span> <span class="n">expression</span>
|
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -425,23 +433,23 @@ conversions rely on type inference.</p>
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#ensure_bools"></a>
|
<a class="headerlink" href="#ensure_bools"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-88"><a href="#ensure_bools-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-92"><a href="#ensure_bools-92"><span class="linenos"> 92</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
|
||||||
</span><span id="ensure_bools-89"><a href="#ensure_bools-89"><span class="linenos"> 89</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
</span><span id="ensure_bools-93"><a href="#ensure_bools-93"><span class="linenos"> 93</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
|
||||||
</span><span id="ensure_bools-90"><a href="#ensure_bools-90"><span class="linenos"> 90</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
</span><span id="ensure_bools-94"><a href="#ensure_bools-94"><span class="linenos"> 94</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="ensure_bools-91"><a href="#ensure_bools-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
</span><span id="ensure_bools-95"><a href="#ensure_bools-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||||
</span><span id="ensure_bools-92"><a href="#ensure_bools-92"><span class="linenos"> 92</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
</span><span id="ensure_bools-96"><a href="#ensure_bools-96"><span class="linenos"> 96</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
|
||||||
</span><span id="ensure_bools-93"><a href="#ensure_bools-93"><span class="linenos"> 93</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
</span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
|
||||||
</span><span id="ensure_bools-94"><a href="#ensure_bools-94"><span class="linenos"> 94</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
</span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
|
||||||
</span><span id="ensure_bools-95"><a href="#ensure_bools-95"><span class="linenos"> 95</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
</span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||||
</span><span id="ensure_bools-96"><a href="#ensure_bools-96"><span class="linenos"> 96</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
</span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="c1"># We can't replace num in CASE x WHEN num ..., because it's not the full predicate</span>
|
||||||
</span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||||
</span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
|
||||||
</span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a> <span class="p">):</span>
|
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a> <span class="p">):</span>
|
||||||
</span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||||
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
</span><span id="ensure_bools-105"><a href="#ensure_bools-105"><span class="linenos">105</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
|
||||||
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
</span><span id="ensure_bools-106"><a href="#ensure_bools-106"><span class="linenos">106</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||||
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a>
|
</span><span id="ensure_bools-107"><a href="#ensure_bools-107"><span class="linenos">107</span></a>
|
||||||
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="k">return</span> <span class="n">expression</span>
|
</span><span id="ensure_bools-108"><a href="#ensure_bools-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -459,12 +467,12 @@ conversions rely on type inference.</p>
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
<a class="headerlink" href="#remove_ascending_order"></a>
|
<a class="headerlink" href="#remove_ascending_order"></a>
|
||||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-107"><a href="#remove_ascending_order-107"><span class="linenos">107</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-111"><a href="#remove_ascending_order-111"><span class="linenos">111</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||||
</span><span id="remove_ascending_order-108"><a href="#remove_ascending_order-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
</span><span id="remove_ascending_order-112"><a href="#remove_ascending_order-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
|
||||||
</span><span id="remove_ascending_order-109"><a href="#remove_ascending_order-109"><span class="linenos">109</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
</span><span id="remove_ascending_order-113"><a href="#remove_ascending_order-113"><span class="linenos">113</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
|
||||||
</span><span id="remove_ascending_order-110"><a href="#remove_ascending_order-110"><span class="linenos">110</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
</span><span id="remove_ascending_order-114"><a href="#remove_ascending_order-114"><span class="linenos">114</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"desc"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||||
</span><span id="remove_ascending_order-111"><a href="#remove_ascending_order-111"><span class="linenos">111</span></a>
|
</span><span id="remove_ascending_order-115"><a href="#remove_ascending_order-115"><span class="linenos">115</span></a>
|
||||||
</span><span id="remove_ascending_order-112"><a href="#remove_ascending_order-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">expression</span>
|
</span><span id="remove_ascending_order-116"><a href="#remove_ascending_order-116"><span class="linenos">116</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -585,7 +585,7 @@ queries if it would result in multiple table selects in a single query:</p>
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">UNMERGABLE_ARGS</span> =
|
<span class="name">UNMERGABLE_ARGS</span> =
|
||||||
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'match', 'format', 'having', 'windows', 'settings', 'laterals', 'connect', 'qualify', 'group', 'cluster', 'distribute', 'kind', 'sample', 'sort', 'locks', 'offset', 'with', 'pivots', 'into', 'distinct', 'limit'}</span>
|
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'distinct', 'settings', 'distribute', 'sample', 'pivots', 'laterals', 'into', 'qualify', 'sort', 'cluster', 'limit', 'format', 'connect', 'windows', 'match', 'having', 'offset', 'kind', 'with', 'group', 'locks'}</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
|
|
@ -756,67 +756,72 @@
|
||||||
</span><span id="L-662"><a href="#L-662"><span class="linenos">662</span></a> <span class="k">else</span><span class="p">:</span>
|
</span><span id="L-662"><a href="#L-662"><span class="linenos">662</span></a> <span class="k">else</span><span class="p">:</span>
|
||||||
</span><span id="L-663"><a href="#L-663"><span class="linenos">663</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
|
</span><span id="L-663"><a href="#L-663"><span class="linenos">663</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
|
||||||
</span><span id="L-664"><a href="#L-664"><span class="linenos">664</span></a>
|
</span><span id="L-664"><a href="#L-664"><span class="linenos">664</span></a>
|
||||||
</span><span id="L-665"><a href="#L-665"><span class="linenos">665</span></a> <span class="c1"># If the source's columns are aliased, their aliases shadow the corresponding column names</span>
|
</span><span id="L-665"><a href="#L-665"><span class="linenos">665</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
|
||||||
</span><span id="L-666"><a href="#L-666"><span class="linenos">666</span></a> <span class="k">return</span> <span class="p">[</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)]</span>
|
</span><span id="L-666"><a href="#L-666"><span class="linenos">666</span></a> <span class="c1"># If the source's columns are aliased, their aliases shadow the corresponding column names.</span>
|
||||||
</span><span id="L-667"><a href="#L-667"><span class="linenos">667</span></a>
|
</span><span id="L-667"><a href="#L-667"><span class="linenos">667</span></a> <span class="c1"># This can be expensive if there are lots of columns, so only do this if column_aliases exist.</span>
|
||||||
</span><span id="L-668"><a href="#L-668"><span class="linenos">668</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
</span><span id="L-668"><a href="#L-668"><span class="linenos">668</span></a> <span class="k">return</span> <span class="p">[</span>
|
||||||
</span><span id="L-669"><a href="#L-669"><span class="linenos">669</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
</span><span id="L-669"><a href="#L-669"><span class="linenos">669</span></a> <span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
|
||||||
</span><span id="L-670"><a href="#L-670"><span class="linenos">670</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
</span><span id="L-670"><a href="#L-670"><span class="linenos">670</span></a> <span class="p">]</span>
|
||||||
</span><span id="L-671"><a href="#L-671"><span class="linenos">671</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
|
</span><span id="L-671"><a href="#L-671"><span class="linenos">671</span></a> <span class="k">return</span> <span class="n">columns</span>
|
||||||
</span><span id="L-672"><a href="#L-672"><span class="linenos">672</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
|
</span><span id="L-672"><a href="#L-672"><span class="linenos">672</span></a>
|
||||||
</span><span id="L-673"><a href="#L-673"><span class="linenos">673</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
</span><span id="L-673"><a href="#L-673"><span class="linenos">673</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||||
</span><span id="L-674"><a href="#L-674"><span class="linenos">674</span></a> <span class="p">)</span>
|
</span><span id="L-674"><a href="#L-674"><span class="linenos">674</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||||
</span><span id="L-675"><a href="#L-675"><span class="linenos">675</span></a> <span class="p">}</span>
|
</span><span id="L-675"><a href="#L-675"><span class="linenos">675</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||||
</span><span id="L-676"><a href="#L-676"><span class="linenos">676</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
</span><span id="L-676"><a href="#L-676"><span class="linenos">676</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
|
||||||
</span><span id="L-677"><a href="#L-677"><span class="linenos">677</span></a>
|
</span><span id="L-677"><a href="#L-677"><span class="linenos">677</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
|
||||||
</span><span id="L-678"><a href="#L-678"><span class="linenos">678</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
|
</span><span id="L-678"><a href="#L-678"><span class="linenos">678</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||||
</span><span id="L-679"><a href="#L-679"><span class="linenos">679</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
|
</span><span id="L-679"><a href="#L-679"><span class="linenos">679</span></a> <span class="p">)</span>
|
||||||
</span><span id="L-680"><a href="#L-680"><span class="linenos">680</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
|
</span><span id="L-680"><a href="#L-680"><span class="linenos">680</span></a> <span class="p">}</span>
|
||||||
</span><span id="L-681"><a href="#L-681"><span class="linenos">681</span></a><span class="w"> </span><span class="sd">"""</span>
|
</span><span id="L-681"><a href="#L-681"><span class="linenos">681</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
||||||
</span><span id="L-682"><a href="#L-682"><span class="linenos">682</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
</span><span id="L-682"><a href="#L-682"><span class="linenos">682</span></a>
|
||||||
</span><span id="L-683"><a href="#L-683"><span class="linenos">683</span></a>
|
</span><span id="L-683"><a href="#L-683"><span class="linenos">683</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
|
||||||
</span><span id="L-684"><a href="#L-684"><span class="linenos">684</span></a><span class="sd"> Args:</span>
|
</span><span id="L-684"><a href="#L-684"><span class="linenos">684</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
|
||||||
</span><span id="L-685"><a href="#L-685"><span class="linenos">685</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
|
</span><span id="L-685"><a href="#L-685"><span class="linenos">685</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
|
||||||
</span><span id="L-686"><a href="#L-686"><span class="linenos">686</span></a>
|
</span><span id="L-686"><a href="#L-686"><span class="linenos">686</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||||
</span><span id="L-687"><a href="#L-687"><span class="linenos">687</span></a><span class="sd"> Returns:</span>
|
</span><span id="L-687"><a href="#L-687"><span class="linenos">687</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
||||||
</span><span id="L-688"><a href="#L-688"><span class="linenos">688</span></a><span class="sd"> Mapping of column name to source name.</span>
|
</span><span id="L-688"><a href="#L-688"><span class="linenos">688</span></a>
|
||||||
</span><span id="L-689"><a href="#L-689"><span class="linenos">689</span></a><span class="sd"> """</span>
|
</span><span id="L-689"><a href="#L-689"><span class="linenos">689</span></a><span class="sd"> Args:</span>
|
||||||
</span><span id="L-690"><a href="#L-690"><span class="linenos">690</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
</span><span id="L-690"><a href="#L-690"><span class="linenos">690</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
|
||||||
</span><span id="L-691"><a href="#L-691"><span class="linenos">691</span></a> <span class="k">return</span> <span class="p">{}</span>
|
</span><span id="L-691"><a href="#L-691"><span class="linenos">691</span></a>
|
||||||
</span><span id="L-692"><a href="#L-692"><span class="linenos">692</span></a>
|
</span><span id="L-692"><a href="#L-692"><span class="linenos">692</span></a><span class="sd"> Returns:</span>
|
||||||
</span><span id="L-693"><a href="#L-693"><span class="linenos">693</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
</span><span id="L-693"><a href="#L-693"><span class="linenos">693</span></a><span class="sd"> Mapping of column name to source name.</span>
|
||||||
</span><span id="L-694"><a href="#L-694"><span class="linenos">694</span></a>
|
</span><span id="L-694"><a href="#L-694"><span class="linenos">694</span></a><span class="sd"> """</span>
|
||||||
</span><span id="L-695"><a href="#L-695"><span class="linenos">695</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
</span><span id="L-695"><a href="#L-695"><span class="linenos">695</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
||||||
</span><span id="L-696"><a href="#L-696"><span class="linenos">696</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
</span><span id="L-696"><a href="#L-696"><span class="linenos">696</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||||
</span><span id="L-697"><a href="#L-697"><span class="linenos">697</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
</span><span id="L-697"><a href="#L-697"><span class="linenos">697</span></a>
|
||||||
</span><span id="L-698"><a href="#L-698"><span class="linenos">698</span></a>
|
</span><span id="L-698"><a href="#L-698"><span class="linenos">698</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
||||||
</span><span id="L-699"><a href="#L-699"><span class="linenos">699</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
</span><span id="L-699"><a href="#L-699"><span class="linenos">699</span></a>
|
||||||
</span><span id="L-700"><a href="#L-700"><span class="linenos">700</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
</span><span id="L-700"><a href="#L-700"><span class="linenos">700</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||||
</span><span id="L-701"><a href="#L-701"><span class="linenos">701</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
</span><span id="L-701"><a href="#L-701"><span class="linenos">701</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
||||||
</span><span id="L-702"><a href="#L-702"><span class="linenos">702</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
</span><span id="L-702"><a href="#L-702"><span class="linenos">702</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
||||||
</span><span id="L-703"><a href="#L-703"><span class="linenos">703</span></a>
|
</span><span id="L-703"><a href="#L-703"><span class="linenos">703</span></a>
|
||||||
</span><span id="L-704"><a href="#L-704"><span class="linenos">704</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
</span><span id="L-704"><a href="#L-704"><span class="linenos">704</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||||
</span><span id="L-705"><a href="#L-705"><span class="linenos">705</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
</span><span id="L-705"><a href="#L-705"><span class="linenos">705</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||||
</span><span id="L-706"><a href="#L-706"><span class="linenos">706</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
</span><span id="L-706"><a href="#L-706"><span class="linenos">706</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
||||||
</span><span id="L-707"><a href="#L-707"><span class="linenos">707</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
</span><span id="L-707"><a href="#L-707"><span class="linenos">707</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||||
</span><span id="L-708"><a href="#L-708"><span class="linenos">708</span></a>
|
</span><span id="L-708"><a href="#L-708"><span class="linenos">708</span></a>
|
||||||
</span><span id="L-709"><a href="#L-709"><span class="linenos">709</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
</span><span id="L-709"><a href="#L-709"><span class="linenos">709</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
||||||
</span><span id="L-710"><a href="#L-710"><span class="linenos">710</span></a>
|
</span><span id="L-710"><a href="#L-710"><span class="linenos">710</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||||
</span><span id="L-711"><a href="#L-711"><span class="linenos">711</span></a> <span class="nd">@staticmethod</span>
|
</span><span id="L-711"><a href="#L-711"><span class="linenos">711</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
||||||
</span><span id="L-712"><a href="#L-712"><span class="linenos">712</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
</span><span id="L-712"><a href="#L-712"><span class="linenos">712</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||||
</span><span id="L-713"><a href="#L-713"><span class="linenos">713</span></a><span class="w"> </span><span class="sd">"""</span>
|
</span><span id="L-713"><a href="#L-713"><span class="linenos">713</span></a>
|
||||||
</span><span id="L-714"><a href="#L-714"><span class="linenos">714</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
</span><span id="L-714"><a href="#L-714"><span class="linenos">714</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
||||||
</span><span id="L-715"><a href="#L-715"><span class="linenos">715</span></a>
|
</span><span id="L-715"><a href="#L-715"><span class="linenos">715</span></a>
|
||||||
</span><span id="L-716"><a href="#L-716"><span class="linenos">716</span></a><span class="sd"> Example:</span>
|
</span><span id="L-716"><a href="#L-716"><span class="linenos">716</span></a> <span class="nd">@staticmethod</span>
|
||||||
</span><span id="L-717"><a href="#L-717"><span class="linenos">717</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
</span><span id="L-717"><a href="#L-717"><span class="linenos">717</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||||
</span><span id="L-718"><a href="#L-718"><span class="linenos">718</span></a><span class="sd"> ['a', 'c']</span>
|
</span><span id="L-718"><a href="#L-718"><span class="linenos">718</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||||
</span><span id="L-719"><a href="#L-719"><span class="linenos">719</span></a>
|
</span><span id="L-719"><a href="#L-719"><span class="linenos">719</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
||||||
</span><span id="L-720"><a href="#L-720"><span class="linenos">720</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
</span><span id="L-720"><a href="#L-720"><span class="linenos">720</span></a>
|
||||||
</span><span id="L-721"><a href="#L-721"><span class="linenos">721</span></a><span class="sd"> """</span>
|
</span><span id="L-721"><a href="#L-721"><span class="linenos">721</span></a><span class="sd"> Example:</span>
|
||||||
</span><span id="L-722"><a href="#L-722"><span class="linenos">722</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
|
</span><span id="L-722"><a href="#L-722"><span class="linenos">722</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
||||||
</span><span id="L-723"><a href="#L-723"><span class="linenos">723</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
</span><span id="L-723"><a href="#L-723"><span class="linenos">723</span></a><span class="sd"> ['a', 'c']</span>
|
||||||
</span><span id="L-724"><a href="#L-724"><span class="linenos">724</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
</span><span id="L-724"><a href="#L-724"><span class="linenos">724</span></a>
|
||||||
</span><span id="L-725"><a href="#L-725"><span class="linenos">725</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
</span><span id="L-725"><a href="#L-725"><span class="linenos">725</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
||||||
|
</span><span id="L-726"><a href="#L-726"><span class="linenos">726</span></a><span class="sd"> """</span>
|
||||||
|
</span><span id="L-727"><a href="#L-727"><span class="linenos">727</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
|
||||||
|
</span><span id="L-728"><a href="#L-728"><span class="linenos">728</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
||||||
|
</span><span id="L-729"><a href="#L-729"><span class="linenos">729</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||||
|
</span><span id="L-730"><a href="#L-730"><span class="linenos">730</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1228,67 +1233,72 @@ know what you're doing!</li>
|
||||||
</span><span id="Resolver-663"><a href="#Resolver-663"><span class="linenos">663</span></a> <span class="k">else</span><span class="p">:</span>
|
</span><span id="Resolver-663"><a href="#Resolver-663"><span class="linenos">663</span></a> <span class="k">else</span><span class="p">:</span>
|
||||||
</span><span id="Resolver-664"><a href="#Resolver-664"><span class="linenos">664</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
|
</span><span id="Resolver-664"><a href="#Resolver-664"><span class="linenos">664</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
|
||||||
</span><span id="Resolver-665"><a href="#Resolver-665"><span class="linenos">665</span></a>
|
</span><span id="Resolver-665"><a href="#Resolver-665"><span class="linenos">665</span></a>
|
||||||
</span><span id="Resolver-666"><a href="#Resolver-666"><span class="linenos">666</span></a> <span class="c1"># If the source's columns are aliased, their aliases shadow the corresponding column names</span>
|
</span><span id="Resolver-666"><a href="#Resolver-666"><span class="linenos">666</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
|
||||||
</span><span id="Resolver-667"><a href="#Resolver-667"><span class="linenos">667</span></a> <span class="k">return</span> <span class="p">[</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)]</span>
|
</span><span id="Resolver-667"><a href="#Resolver-667"><span class="linenos">667</span></a> <span class="c1"># If the source's columns are aliased, their aliases shadow the corresponding column names.</span>
|
||||||
</span><span id="Resolver-668"><a href="#Resolver-668"><span class="linenos">668</span></a>
|
</span><span id="Resolver-668"><a href="#Resolver-668"><span class="linenos">668</span></a> <span class="c1"># This can be expensive if there are lots of columns, so only do this if column_aliases exist.</span>
|
||||||
</span><span id="Resolver-669"><a href="#Resolver-669"><span class="linenos">669</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
</span><span id="Resolver-669"><a href="#Resolver-669"><span class="linenos">669</span></a> <span class="k">return</span> <span class="p">[</span>
|
||||||
</span><span id="Resolver-670"><a href="#Resolver-670"><span class="linenos">670</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
</span><span id="Resolver-670"><a href="#Resolver-670"><span class="linenos">670</span></a> <span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-671"><a href="#Resolver-671"><span class="linenos">671</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
</span><span id="Resolver-671"><a href="#Resolver-671"><span class="linenos">671</span></a> <span class="p">]</span>
|
||||||
</span><span id="Resolver-672"><a href="#Resolver-672"><span class="linenos">672</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
|
</span><span id="Resolver-672"><a href="#Resolver-672"><span class="linenos">672</span></a> <span class="k">return</span> <span class="n">columns</span>
|
||||||
</span><span id="Resolver-673"><a href="#Resolver-673"><span class="linenos">673</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
|
</span><span id="Resolver-673"><a href="#Resolver-673"><span class="linenos">673</span></a>
|
||||||
</span><span id="Resolver-674"><a href="#Resolver-674"><span class="linenos">674</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
</span><span id="Resolver-674"><a href="#Resolver-674"><span class="linenos">674</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||||
</span><span id="Resolver-675"><a href="#Resolver-675"><span class="linenos">675</span></a> <span class="p">)</span>
|
</span><span id="Resolver-675"><a href="#Resolver-675"><span class="linenos">675</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||||
</span><span id="Resolver-676"><a href="#Resolver-676"><span class="linenos">676</span></a> <span class="p">}</span>
|
</span><span id="Resolver-676"><a href="#Resolver-676"><span class="linenos">676</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||||
</span><span id="Resolver-677"><a href="#Resolver-677"><span class="linenos">677</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
</span><span id="Resolver-677"><a href="#Resolver-677"><span class="linenos">677</span></a> <span class="n">source_name</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">source_name</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-678"><a href="#Resolver-678"><span class="linenos">678</span></a>
|
</span><span id="Resolver-678"><a href="#Resolver-678"><span class="linenos">678</span></a> <span class="k">for</span> <span class="n">source_name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span>
|
||||||
</span><span id="Resolver-679"><a href="#Resolver-679"><span class="linenos">679</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
|
</span><span id="Resolver-679"><a href="#Resolver-679"><span class="linenos">679</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||||
</span><span id="Resolver-680"><a href="#Resolver-680"><span class="linenos">680</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
|
</span><span id="Resolver-680"><a href="#Resolver-680"><span class="linenos">680</span></a> <span class="p">)</span>
|
||||||
</span><span id="Resolver-681"><a href="#Resolver-681"><span class="linenos">681</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
|
</span><span id="Resolver-681"><a href="#Resolver-681"><span class="linenos">681</span></a> <span class="p">}</span>
|
||||||
</span><span id="Resolver-682"><a href="#Resolver-682"><span class="linenos">682</span></a><span class="w"> </span><span class="sd">"""</span>
|
</span><span id="Resolver-682"><a href="#Resolver-682"><span class="linenos">682</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
||||||
</span><span id="Resolver-683"><a href="#Resolver-683"><span class="linenos">683</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
</span><span id="Resolver-683"><a href="#Resolver-683"><span class="linenos">683</span></a>
|
||||||
</span><span id="Resolver-684"><a href="#Resolver-684"><span class="linenos">684</span></a>
|
</span><span id="Resolver-684"><a href="#Resolver-684"><span class="linenos">684</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span>
|
||||||
</span><span id="Resolver-685"><a href="#Resolver-685"><span class="linenos">685</span></a><span class="sd"> Args:</span>
|
</span><span id="Resolver-685"><a href="#Resolver-685"><span class="linenos">685</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
|
||||||
</span><span id="Resolver-686"><a href="#Resolver-686"><span class="linenos">686</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
|
</span><span id="Resolver-686"><a href="#Resolver-686"><span class="linenos">686</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]:</span>
|
||||||
</span><span id="Resolver-687"><a href="#Resolver-687"><span class="linenos">687</span></a>
|
</span><span id="Resolver-687"><a href="#Resolver-687"><span class="linenos">687</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||||
</span><span id="Resolver-688"><a href="#Resolver-688"><span class="linenos">688</span></a><span class="sd"> Returns:</span>
|
</span><span id="Resolver-688"><a href="#Resolver-688"><span class="linenos">688</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
||||||
</span><span id="Resolver-689"><a href="#Resolver-689"><span class="linenos">689</span></a><span class="sd"> Mapping of column name to source name.</span>
|
</span><span id="Resolver-689"><a href="#Resolver-689"><span class="linenos">689</span></a>
|
||||||
</span><span id="Resolver-690"><a href="#Resolver-690"><span class="linenos">690</span></a><span class="sd"> """</span>
|
</span><span id="Resolver-690"><a href="#Resolver-690"><span class="linenos">690</span></a><span class="sd"> Args:</span>
|
||||||
</span><span id="Resolver-691"><a href="#Resolver-691"><span class="linenos">691</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
</span><span id="Resolver-691"><a href="#Resolver-691"><span class="linenos">691</span></a><span class="sd"> source_columns: Mapping of names to source columns.</span>
|
||||||
</span><span id="Resolver-692"><a href="#Resolver-692"><span class="linenos">692</span></a> <span class="k">return</span> <span class="p">{}</span>
|
</span><span id="Resolver-692"><a href="#Resolver-692"><span class="linenos">692</span></a>
|
||||||
</span><span id="Resolver-693"><a href="#Resolver-693"><span class="linenos">693</span></a>
|
</span><span id="Resolver-693"><a href="#Resolver-693"><span class="linenos">693</span></a><span class="sd"> Returns:</span>
|
||||||
</span><span id="Resolver-694"><a href="#Resolver-694"><span class="linenos">694</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
</span><span id="Resolver-694"><a href="#Resolver-694"><span class="linenos">694</span></a><span class="sd"> Mapping of column name to source name.</span>
|
||||||
</span><span id="Resolver-695"><a href="#Resolver-695"><span class="linenos">695</span></a>
|
</span><span id="Resolver-695"><a href="#Resolver-695"><span class="linenos">695</span></a><span class="sd"> """</span>
|
||||||
</span><span id="Resolver-696"><a href="#Resolver-696"><span class="linenos">696</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
</span><span id="Resolver-696"><a href="#Resolver-696"><span class="linenos">696</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
||||||
</span><span id="Resolver-697"><a href="#Resolver-697"><span class="linenos">697</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
</span><span id="Resolver-697"><a href="#Resolver-697"><span class="linenos">697</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||||
</span><span id="Resolver-698"><a href="#Resolver-698"><span class="linenos">698</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
</span><span id="Resolver-698"><a href="#Resolver-698"><span class="linenos">698</span></a>
|
||||||
</span><span id="Resolver-699"><a href="#Resolver-699"><span class="linenos">699</span></a>
|
</span><span id="Resolver-699"><a href="#Resolver-699"><span class="linenos">699</span></a> <span class="n">source_columns_pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
||||||
</span><span id="Resolver-700"><a href="#Resolver-700"><span class="linenos">700</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
</span><span id="Resolver-700"><a href="#Resolver-700"><span class="linenos">700</span></a>
|
||||||
</span><span id="Resolver-701"><a href="#Resolver-701"><span class="linenos">701</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
</span><span id="Resolver-701"><a href="#Resolver-701"><span class="linenos">701</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||||
</span><span id="Resolver-702"><a href="#Resolver-702"><span class="linenos">702</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
</span><span id="Resolver-702"><a href="#Resolver-702"><span class="linenos">702</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
||||||
</span><span id="Resolver-703"><a href="#Resolver-703"><span class="linenos">703</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
</span><span id="Resolver-703"><a href="#Resolver-703"><span class="linenos">703</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-704"><a href="#Resolver-704"><span class="linenos">704</span></a>
|
</span><span id="Resolver-704"><a href="#Resolver-704"><span class="linenos">704</span></a>
|
||||||
</span><span id="Resolver-705"><a href="#Resolver-705"><span class="linenos">705</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
</span><span id="Resolver-705"><a href="#Resolver-705"><span class="linenos">705</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns_pairs</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||||
</span><span id="Resolver-706"><a href="#Resolver-706"><span class="linenos">706</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
</span><span id="Resolver-706"><a href="#Resolver-706"><span class="linenos">706</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-707"><a href="#Resolver-707"><span class="linenos">707</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
</span><span id="Resolver-707"><a href="#Resolver-707"><span class="linenos">707</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-708"><a href="#Resolver-708"><span class="linenos">708</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
</span><span id="Resolver-708"><a href="#Resolver-708"><span class="linenos">708</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-709"><a href="#Resolver-709"><span class="linenos">709</span></a>
|
</span><span id="Resolver-709"><a href="#Resolver-709"><span class="linenos">709</span></a>
|
||||||
</span><span id="Resolver-710"><a href="#Resolver-710"><span class="linenos">710</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
</span><span id="Resolver-710"><a href="#Resolver-710"><span class="linenos">710</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
||||||
</span><span id="Resolver-711"><a href="#Resolver-711"><span class="linenos">711</span></a>
|
</span><span id="Resolver-711"><a href="#Resolver-711"><span class="linenos">711</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||||
</span><span id="Resolver-712"><a href="#Resolver-712"><span class="linenos">712</span></a> <span class="nd">@staticmethod</span>
|
</span><span id="Resolver-712"><a href="#Resolver-712"><span class="linenos">712</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
||||||
</span><span id="Resolver-713"><a href="#Resolver-713"><span class="linenos">713</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
</span><span id="Resolver-713"><a href="#Resolver-713"><span class="linenos">713</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||||
</span><span id="Resolver-714"><a href="#Resolver-714"><span class="linenos">714</span></a><span class="w"> </span><span class="sd">"""</span>
|
</span><span id="Resolver-714"><a href="#Resolver-714"><span class="linenos">714</span></a>
|
||||||
</span><span id="Resolver-715"><a href="#Resolver-715"><span class="linenos">715</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
</span><span id="Resolver-715"><a href="#Resolver-715"><span class="linenos">715</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
||||||
</span><span id="Resolver-716"><a href="#Resolver-716"><span class="linenos">716</span></a>
|
</span><span id="Resolver-716"><a href="#Resolver-716"><span class="linenos">716</span></a>
|
||||||
</span><span id="Resolver-717"><a href="#Resolver-717"><span class="linenos">717</span></a><span class="sd"> Example:</span>
|
</span><span id="Resolver-717"><a href="#Resolver-717"><span class="linenos">717</span></a> <span class="nd">@staticmethod</span>
|
||||||
</span><span id="Resolver-718"><a href="#Resolver-718"><span class="linenos">718</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
</span><span id="Resolver-718"><a href="#Resolver-718"><span class="linenos">718</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||||
</span><span id="Resolver-719"><a href="#Resolver-719"><span class="linenos">719</span></a><span class="sd"> ['a', 'c']</span>
|
</span><span id="Resolver-719"><a href="#Resolver-719"><span class="linenos">719</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||||
</span><span id="Resolver-720"><a href="#Resolver-720"><span class="linenos">720</span></a>
|
</span><span id="Resolver-720"><a href="#Resolver-720"><span class="linenos">720</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
||||||
</span><span id="Resolver-721"><a href="#Resolver-721"><span class="linenos">721</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
</span><span id="Resolver-721"><a href="#Resolver-721"><span class="linenos">721</span></a>
|
||||||
</span><span id="Resolver-722"><a href="#Resolver-722"><span class="linenos">722</span></a><span class="sd"> """</span>
|
</span><span id="Resolver-722"><a href="#Resolver-722"><span class="linenos">722</span></a><span class="sd"> Example:</span>
|
||||||
</span><span id="Resolver-723"><a href="#Resolver-723"><span class="linenos">723</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
|
</span><span id="Resolver-723"><a href="#Resolver-723"><span class="linenos">723</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
||||||
</span><span id="Resolver-724"><a href="#Resolver-724"><span class="linenos">724</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
</span><span id="Resolver-724"><a href="#Resolver-724"><span class="linenos">724</span></a><span class="sd"> ['a', 'c']</span>
|
||||||
</span><span id="Resolver-725"><a href="#Resolver-725"><span class="linenos">725</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
</span><span id="Resolver-725"><a href="#Resolver-725"><span class="linenos">725</span></a>
|
||||||
</span><span id="Resolver-726"><a href="#Resolver-726"><span class="linenos">726</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
</span><span id="Resolver-726"><a href="#Resolver-726"><span class="linenos">726</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
||||||
|
</span><span id="Resolver-727"><a href="#Resolver-727"><span class="linenos">727</span></a><span class="sd"> """</span>
|
||||||
|
</span><span id="Resolver-728"><a href="#Resolver-728"><span class="linenos">728</span></a> <span class="n">counts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="p">{}</span>
|
||||||
|
</span><span id="Resolver-729"><a href="#Resolver-729"><span class="linenos">729</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
||||||
|
</span><span id="Resolver-730"><a href="#Resolver-730"><span class="linenos">730</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||||
|
</span><span id="Resolver-731"><a href="#Resolver-731"><span class="linenos">731</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1471,8 +1481,13 @@ know what you're doing!</li>
|
||||||
</span><span id="Resolver.get_source_columns-663"><a href="#Resolver.get_source_columns-663"><span class="linenos">663</span></a> <span class="k">else</span><span class="p">:</span>
|
</span><span id="Resolver.get_source_columns-663"><a href="#Resolver.get_source_columns-663"><span class="linenos">663</span></a> <span class="k">else</span><span class="p">:</span>
|
||||||
</span><span id="Resolver.get_source_columns-664"><a href="#Resolver.get_source_columns-664"><span class="linenos">664</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
|
</span><span id="Resolver.get_source_columns-664"><a href="#Resolver.get_source_columns-664"><span class="linenos">664</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="p">[]</span>
|
||||||
</span><span id="Resolver.get_source_columns-665"><a href="#Resolver.get_source_columns-665"><span class="linenos">665</span></a>
|
</span><span id="Resolver.get_source_columns-665"><a href="#Resolver.get_source_columns-665"><span class="linenos">665</span></a>
|
||||||
</span><span id="Resolver.get_source_columns-666"><a href="#Resolver.get_source_columns-666"><span class="linenos">666</span></a> <span class="c1"># If the source's columns are aliased, their aliases shadow the corresponding column names</span>
|
</span><span id="Resolver.get_source_columns-666"><a href="#Resolver.get_source_columns-666"><span class="linenos">666</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
|
||||||
</span><span id="Resolver.get_source_columns-667"><a href="#Resolver.get_source_columns-667"><span class="linenos">667</span></a> <span class="k">return</span> <span class="p">[</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)]</span>
|
</span><span id="Resolver.get_source_columns-667"><a href="#Resolver.get_source_columns-667"><span class="linenos">667</span></a> <span class="c1"># If the source's columns are aliased, their aliases shadow the corresponding column names.</span>
|
||||||
|
</span><span id="Resolver.get_source_columns-668"><a href="#Resolver.get_source_columns-668"><span class="linenos">668</span></a> <span class="c1"># This can be expensive if there are lots of columns, so only do this if column_aliases exist.</span>
|
||||||
|
</span><span id="Resolver.get_source_columns-669"><a href="#Resolver.get_source_columns-669"><span class="linenos">669</span></a> <span class="k">return</span> <span class="p">[</span>
|
||||||
|
</span><span id="Resolver.get_source_columns-670"><a href="#Resolver.get_source_columns-670"><span class="linenos">670</span></a> <span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span> <span class="k">for</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">columns</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
|
||||||
|
</span><span id="Resolver.get_source_columns-671"><a href="#Resolver.get_source_columns-671"><span class="linenos">671</span></a> <span class="p">]</span>
|
||||||
|
</span><span id="Resolver.get_source_columns-672"><a href="#Resolver.get_source_columns-672"><span class="linenos">672</span></a> <span class="k">return</span> <span class="n">columns</span>
|
||||||
</span></pre></div>
|
</span></pre></div>
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2546,7 +2546,7 @@ prefix are statically known.</p>
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">DATETRUNC_COMPARISONS</span> =
|
<span class="name">DATETRUNC_COMPARISONS</span> =
|
||||||
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||||
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{<class '<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>'>, <class '<a href="../expressions.html#GT">sqlglot.expressions.GT</a>'>, <class '<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>'>, <class '<a href="../expressions.html#LT">sqlglot.expressions.LT</a>'>, <class '<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>'>, <class '<a href="../expressions.html#In">sqlglot.expressions.In</a>'>, <class '<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>'>}</span>
|
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{<class '<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>'>, <class '<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>'>, <class '<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>'>, <class '<a href="../expressions.html#GT">sqlglot.expressions.GT</a>'>, <class '<a href="../expressions.html#LT">sqlglot.expressions.LT</a>'>, <class '<a href="../expressions.html#In">sqlglot.expressions.In</a>'>, <class '<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>'>}</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
@ -2626,7 +2626,7 @@ prefix are statically known.</p>
|
||||||
<section id="JOINS">
|
<section id="JOINS">
|
||||||
<div class="attr variable">
|
<div class="attr variable">
|
||||||
<span class="name">JOINS</span> =
|
<span class="name">JOINS</span> =
|
||||||
<span class="default_value">{('', ''), ('', 'INNER'), ('RIGHT', ''), ('RIGHT', 'OUTER')}</span>
|
<span class="default_value">{('', 'INNER'), ('RIGHT', 'OUTER'), ('RIGHT', ''), ('', '')}</span>
|
||||||
|
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
20986
docs/sqlglot/parser.html
20986
docs/sqlglot/parser.html
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -148,7 +148,7 @@ def atanh(col: ColumnOrName) -> Column:
|
||||||
|
|
||||||
|
|
||||||
def cbrt(col: ColumnOrName) -> Column:
|
def cbrt(col: ColumnOrName) -> Column:
|
||||||
return Column.invoke_anonymous_function(col, "CBRT")
|
return Column.invoke_expression_over_column(col, expression.Cbrt)
|
||||||
|
|
||||||
|
|
||||||
def ceil(col: ColumnOrName) -> Column:
|
def ceil(col: ColumnOrName) -> Column:
|
||||||
|
|
|
||||||
|
|
@ -70,14 +70,12 @@ class SparkSession:
|
||||||
column_mapping = {f"_{i}": None for i in range(1, len(data[0]) + 1)}
|
column_mapping = {f"_{i}": None for i in range(1, len(data[0]) + 1)}
|
||||||
|
|
||||||
data_expressions = [
|
data_expressions = [
|
||||||
exp.Tuple(
|
exp.tuple_(
|
||||||
expressions=list(
|
*map(
|
||||||
map(
|
|
||||||
lambda x: F.lit(x).expression,
|
lambda x: F.lit(x).expression,
|
||||||
row if not isinstance(row, dict) else row.values(),
|
row if not isinstance(row, dict) else row.values(),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
)
|
|
||||||
for row in data
|
for row in data
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,11 @@ def _derived_table_values_to_unnest(self: BigQuery.Generator, expression: exp.Va
|
||||||
|
|
||||||
alias = expression.args.get("alias")
|
alias = expression.args.get("alias")
|
||||||
|
|
||||||
structs = [
|
return self.unnest_sql(
|
||||||
|
exp.Unnest(
|
||||||
|
expressions=[
|
||||||
|
exp.array(
|
||||||
|
*(
|
||||||
exp.Struct(
|
exp.Struct(
|
||||||
expressions=[
|
expressions=[
|
||||||
exp.alias_(value, column_name)
|
exp.alias_(value, column_name)
|
||||||
|
|
@ -54,9 +58,12 @@ def _derived_table_values_to_unnest(self: BigQuery.Generator, expression: exp.Va
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
for t in expression.find_all(exp.Tuple)
|
for t in expression.find_all(exp.Tuple)
|
||||||
|
),
|
||||||
|
copy=False,
|
||||||
|
)
|
||||||
]
|
]
|
||||||
|
)
|
||||||
return self.unnest_sql(exp.Unnest(expressions=[exp.Array(expressions=structs)]))
|
)
|
||||||
|
|
||||||
|
|
||||||
def _returnsproperty_sql(self: BigQuery.Generator, expression: exp.ReturnsProperty) -> str:
|
def _returnsproperty_sql(self: BigQuery.Generator, expression: exp.ReturnsProperty) -> str:
|
||||||
|
|
@ -161,12 +168,18 @@ def _pushdown_cte_column_names(expression: exp.Expression) -> exp.Expression:
|
||||||
return expression
|
return expression
|
||||||
|
|
||||||
|
|
||||||
def _parse_timestamp(args: t.List) -> exp.StrToTime:
|
def _parse_parse_timestamp(args: t.List) -> exp.StrToTime:
|
||||||
this = format_time_lambda(exp.StrToTime, "bigquery")([seq_get(args, 1), seq_get(args, 0)])
|
this = format_time_lambda(exp.StrToTime, "bigquery")([seq_get(args, 1), seq_get(args, 0)])
|
||||||
this.set("zone", seq_get(args, 2))
|
this.set("zone", seq_get(args, 2))
|
||||||
return this
|
return this
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_timestamp(args: t.List) -> exp.Timestamp:
|
||||||
|
timestamp = exp.Timestamp.from_arg_list(args)
|
||||||
|
timestamp.set("with_tz", True)
|
||||||
|
return timestamp
|
||||||
|
|
||||||
|
|
||||||
def _parse_date(args: t.List) -> exp.Date | exp.DateFromParts:
|
def _parse_date(args: t.List) -> exp.Date | exp.DateFromParts:
|
||||||
expr_type = exp.DateFromParts if len(args) == 3 else exp.Date
|
expr_type = exp.DateFromParts if len(args) == 3 else exp.Date
|
||||||
return expr_type.from_arg_list(args)
|
return expr_type.from_arg_list(args)
|
||||||
|
|
@ -318,6 +331,7 @@ class BigQuery(Dialect):
|
||||||
"TIMESTAMP": TokenType.TIMESTAMPTZ,
|
"TIMESTAMP": TokenType.TIMESTAMPTZ,
|
||||||
}
|
}
|
||||||
KEYWORDS.pop("DIV")
|
KEYWORDS.pop("DIV")
|
||||||
|
KEYWORDS.pop("VALUES")
|
||||||
|
|
||||||
class Parser(parser.Parser):
|
class Parser(parser.Parser):
|
||||||
PREFIXED_PIVOT_COLUMNS = True
|
PREFIXED_PIVOT_COLUMNS = True
|
||||||
|
|
@ -348,7 +362,7 @@ class BigQuery(Dialect):
|
||||||
"PARSE_DATE": lambda args: format_time_lambda(exp.StrToDate, "bigquery")(
|
"PARSE_DATE": lambda args: format_time_lambda(exp.StrToDate, "bigquery")(
|
||||||
[seq_get(args, 1), seq_get(args, 0)]
|
[seq_get(args, 1), seq_get(args, 0)]
|
||||||
),
|
),
|
||||||
"PARSE_TIMESTAMP": _parse_timestamp,
|
"PARSE_TIMESTAMP": _parse_parse_timestamp,
|
||||||
"REGEXP_CONTAINS": exp.RegexpLike.from_arg_list,
|
"REGEXP_CONTAINS": exp.RegexpLike.from_arg_list,
|
||||||
"REGEXP_EXTRACT": lambda args: exp.RegexpExtract(
|
"REGEXP_EXTRACT": lambda args: exp.RegexpExtract(
|
||||||
this=seq_get(args, 0),
|
this=seq_get(args, 0),
|
||||||
|
|
@ -367,6 +381,7 @@ class BigQuery(Dialect):
|
||||||
"TIME": _parse_time,
|
"TIME": _parse_time,
|
||||||
"TIME_ADD": parse_date_delta_with_interval(exp.TimeAdd),
|
"TIME_ADD": parse_date_delta_with_interval(exp.TimeAdd),
|
||||||
"TIME_SUB": parse_date_delta_with_interval(exp.TimeSub),
|
"TIME_SUB": parse_date_delta_with_interval(exp.TimeSub),
|
||||||
|
"TIMESTAMP": _parse_timestamp,
|
||||||
"TIMESTAMP_ADD": parse_date_delta_with_interval(exp.TimestampAdd),
|
"TIMESTAMP_ADD": parse_date_delta_with_interval(exp.TimestampAdd),
|
||||||
"TIMESTAMP_SUB": parse_date_delta_with_interval(exp.TimestampSub),
|
"TIMESTAMP_SUB": parse_date_delta_with_interval(exp.TimestampSub),
|
||||||
"TIMESTAMP_MICROS": lambda args: exp.UnixToTime(
|
"TIMESTAMP_MICROS": lambda args: exp.UnixToTime(
|
||||||
|
|
@ -395,11 +410,6 @@ class BigQuery(Dialect):
|
||||||
TokenType.TABLE,
|
TokenType.TABLE,
|
||||||
}
|
}
|
||||||
|
|
||||||
ID_VAR_TOKENS = {
|
|
||||||
*parser.Parser.ID_VAR_TOKENS,
|
|
||||||
TokenType.VALUES,
|
|
||||||
}
|
|
||||||
|
|
||||||
PROPERTY_PARSERS = {
|
PROPERTY_PARSERS = {
|
||||||
**parser.Parser.PROPERTY_PARSERS,
|
**parser.Parser.PROPERTY_PARSERS,
|
||||||
"NOT DETERMINISTIC": lambda self: self.expression(
|
"NOT DETERMINISTIC": lambda self: self.expression(
|
||||||
|
|
|
||||||
|
|
@ -93,6 +93,7 @@ class ClickHouse(Dialect):
|
||||||
"IPV6": TokenType.IPV6,
|
"IPV6": TokenType.IPV6,
|
||||||
"AGGREGATEFUNCTION": TokenType.AGGREGATEFUNCTION,
|
"AGGREGATEFUNCTION": TokenType.AGGREGATEFUNCTION,
|
||||||
"SIMPLEAGGREGATEFUNCTION": TokenType.SIMPLEAGGREGATEFUNCTION,
|
"SIMPLEAGGREGATEFUNCTION": TokenType.SIMPLEAGGREGATEFUNCTION,
|
||||||
|
"SYSTEM": TokenType.COMMAND,
|
||||||
}
|
}
|
||||||
|
|
||||||
SINGLE_TOKENS = {
|
SINGLE_TOKENS = {
|
||||||
|
|
|
||||||
|
|
@ -654,28 +654,6 @@ def time_format(
|
||||||
return _time_format
|
return _time_format
|
||||||
|
|
||||||
|
|
||||||
def create_with_partitions_sql(self: Generator, expression: exp.Create) -> str:
|
|
||||||
"""
|
|
||||||
In Hive and Spark, the PARTITIONED BY property acts as an extension of a table's schema. When the
|
|
||||||
PARTITIONED BY value is an array of column names, they are transformed into a schema. The corresponding
|
|
||||||
columns are removed from the create statement.
|
|
||||||
"""
|
|
||||||
has_schema = isinstance(expression.this, exp.Schema)
|
|
||||||
is_partitionable = expression.args.get("kind") in ("TABLE", "VIEW")
|
|
||||||
|
|
||||||
if has_schema and is_partitionable:
|
|
||||||
prop = expression.find(exp.PartitionedByProperty)
|
|
||||||
if prop and prop.this and not isinstance(prop.this, exp.Schema):
|
|
||||||
schema = expression.this
|
|
||||||
columns = {v.name.upper() for v in prop.this.expressions}
|
|
||||||
partitions = [col for col in schema.expressions if col.name.upper() in columns]
|
|
||||||
schema.set("expressions", [e for e in schema.expressions if e not in partitions])
|
|
||||||
prop.replace(exp.PartitionedByProperty(this=exp.Schema(expressions=partitions)))
|
|
||||||
expression.set("this", schema)
|
|
||||||
|
|
||||||
return self.create_sql(expression)
|
|
||||||
|
|
||||||
|
|
||||||
def parse_date_delta(
|
def parse_date_delta(
|
||||||
exp_class: t.Type[E], unit_mapping: t.Optional[t.Dict[str, str]] = None
|
exp_class: t.Type[E], unit_mapping: t.Optional[t.Dict[str, str]] = None
|
||||||
) -> t.Callable[[t.List], E]:
|
) -> t.Callable[[t.List], E]:
|
||||||
|
|
@ -742,7 +720,10 @@ def timestamptrunc_sql(self: Generator, expression: exp.TimestampTrunc) -> str:
|
||||||
|
|
||||||
def no_timestamp_sql(self: Generator, expression: exp.Timestamp) -> str:
|
def no_timestamp_sql(self: Generator, expression: exp.Timestamp) -> str:
|
||||||
if not expression.expression:
|
if not expression.expression:
|
||||||
return self.sql(exp.cast(expression.this, to=exp.DataType.Type.TIMESTAMP))
|
from sqlglot.optimizer.annotate_types import annotate_types
|
||||||
|
|
||||||
|
target_type = annotate_types(expression).type or exp.DataType.Type.TIMESTAMP
|
||||||
|
return self.sql(exp.cast(expression.this, to=target_type))
|
||||||
if expression.text("expression").lower() in TIMEZONES:
|
if expression.text("expression").lower() in TIMEZONES:
|
||||||
return self.sql(
|
return self.sql(
|
||||||
exp.AtTimeZone(
|
exp.AtTimeZone(
|
||||||
|
|
@ -750,7 +731,7 @@ def no_timestamp_sql(self: Generator, expression: exp.Timestamp) -> str:
|
||||||
zone=expression.expression,
|
zone=expression.expression,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return self.function_fallback_sql(expression)
|
return self.func("TIMESTAMP", expression.this, expression.expression)
|
||||||
|
|
||||||
|
|
||||||
def locate_to_strposition(args: t.List) -> exp.Expression:
|
def locate_to_strposition(args: t.List) -> exp.Expression:
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,6 @@ import typing as t
|
||||||
from sqlglot import exp, generator, parser, tokens, transforms
|
from sqlglot import exp, generator, parser, tokens, transforms
|
||||||
from sqlglot.dialects.dialect import (
|
from sqlglot.dialects.dialect import (
|
||||||
Dialect,
|
Dialect,
|
||||||
create_with_partitions_sql,
|
|
||||||
datestrtodate_sql,
|
datestrtodate_sql,
|
||||||
format_time_lambda,
|
format_time_lambda,
|
||||||
no_trycast_sql,
|
no_trycast_sql,
|
||||||
|
|
@ -13,6 +12,7 @@ from sqlglot.dialects.dialect import (
|
||||||
str_position_sql,
|
str_position_sql,
|
||||||
timestrtotime_sql,
|
timestrtotime_sql,
|
||||||
)
|
)
|
||||||
|
from sqlglot.transforms import preprocess, move_schema_columns_to_partitioned_by
|
||||||
|
|
||||||
|
|
||||||
def _date_add_sql(kind: str) -> t.Callable[[Drill.Generator, exp.DateAdd | exp.DateSub], str]:
|
def _date_add_sql(kind: str) -> t.Callable[[Drill.Generator, exp.DateAdd | exp.DateSub], str]:
|
||||||
|
|
@ -125,7 +125,7 @@ class Drill(Dialect):
|
||||||
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
|
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
|
||||||
exp.ArrayContains: rename_func("REPEATED_CONTAINS"),
|
exp.ArrayContains: rename_func("REPEATED_CONTAINS"),
|
||||||
exp.ArraySize: rename_func("REPEATED_COUNT"),
|
exp.ArraySize: rename_func("REPEATED_COUNT"),
|
||||||
exp.Create: create_with_partitions_sql,
|
exp.Create: preprocess([move_schema_columns_to_partitioned_by]),
|
||||||
exp.DateAdd: _date_add_sql("ADD"),
|
exp.DateAdd: _date_add_sql("ADD"),
|
||||||
exp.DateStrToDate: datestrtodate_sql,
|
exp.DateStrToDate: datestrtodate_sql,
|
||||||
exp.DateSub: _date_add_sql("SUB"),
|
exp.DateSub: _date_add_sql("SUB"),
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,6 @@ from sqlglot.dialects.dialect import (
|
||||||
NormalizationStrategy,
|
NormalizationStrategy,
|
||||||
approx_count_distinct_sql,
|
approx_count_distinct_sql,
|
||||||
arg_max_or_min_no_count,
|
arg_max_or_min_no_count,
|
||||||
create_with_partitions_sql,
|
|
||||||
datestrtodate_sql,
|
datestrtodate_sql,
|
||||||
format_time_lambda,
|
format_time_lambda,
|
||||||
if_sql,
|
if_sql,
|
||||||
|
|
@ -32,6 +31,12 @@ from sqlglot.dialects.dialect import (
|
||||||
timestrtotime_sql,
|
timestrtotime_sql,
|
||||||
var_map_sql,
|
var_map_sql,
|
||||||
)
|
)
|
||||||
|
from sqlglot.transforms import (
|
||||||
|
remove_unique_constraints,
|
||||||
|
ctas_with_tmp_tables_to_create_tmp_view,
|
||||||
|
preprocess,
|
||||||
|
move_schema_columns_to_partitioned_by,
|
||||||
|
)
|
||||||
from sqlglot.helper import seq_get
|
from sqlglot.helper import seq_get
|
||||||
from sqlglot.parser import parse_var_map
|
from sqlglot.parser import parse_var_map
|
||||||
from sqlglot.tokens import TokenType
|
from sqlglot.tokens import TokenType
|
||||||
|
|
@ -55,30 +60,6 @@ TIME_DIFF_FACTOR = {
|
||||||
DIFF_MONTH_SWITCH = ("YEAR", "QUARTER", "MONTH")
|
DIFF_MONTH_SWITCH = ("YEAR", "QUARTER", "MONTH")
|
||||||
|
|
||||||
|
|
||||||
def _create_sql(self, expression: exp.Create) -> str:
|
|
||||||
# remove UNIQUE column constraints
|
|
||||||
for constraint in expression.find_all(exp.UniqueColumnConstraint):
|
|
||||||
if constraint.parent:
|
|
||||||
constraint.parent.pop()
|
|
||||||
|
|
||||||
properties = expression.args.get("properties")
|
|
||||||
temporary = any(
|
|
||||||
isinstance(prop, exp.TemporaryProperty)
|
|
||||||
for prop in (properties.expressions if properties else [])
|
|
||||||
)
|
|
||||||
|
|
||||||
# CTAS with temp tables map to CREATE TEMPORARY VIEW
|
|
||||||
kind = expression.args["kind"]
|
|
||||||
if kind.upper() == "TABLE" and temporary:
|
|
||||||
if expression.expression:
|
|
||||||
return f"CREATE TEMPORARY VIEW {self.sql(expression, 'this')} AS {self.sql(expression, 'expression')}"
|
|
||||||
else:
|
|
||||||
# CREATE TEMPORARY TABLE may require storage provider
|
|
||||||
expression = self.temporary_storage_provider(expression)
|
|
||||||
|
|
||||||
return create_with_partitions_sql(self, expression)
|
|
||||||
|
|
||||||
|
|
||||||
def _add_date_sql(self: Hive.Generator, expression: DATE_ADD_OR_SUB) -> str:
|
def _add_date_sql(self: Hive.Generator, expression: DATE_ADD_OR_SUB) -> str:
|
||||||
if isinstance(expression, exp.TsOrDsAdd) and not expression.unit:
|
if isinstance(expression, exp.TsOrDsAdd) and not expression.unit:
|
||||||
return self.func("DATE_ADD", expression.this, expression.expression)
|
return self.func("DATE_ADD", expression.this, expression.expression)
|
||||||
|
|
@ -285,6 +266,7 @@ class Hive(Dialect):
|
||||||
class Parser(parser.Parser):
|
class Parser(parser.Parser):
|
||||||
LOG_DEFAULTS_TO_LN = True
|
LOG_DEFAULTS_TO_LN = True
|
||||||
STRICT_CAST = False
|
STRICT_CAST = False
|
||||||
|
VALUES_FOLLOWED_BY_PAREN = False
|
||||||
|
|
||||||
FUNCTIONS = {
|
FUNCTIONS = {
|
||||||
**parser.Parser.FUNCTIONS,
|
**parser.Parser.FUNCTIONS,
|
||||||
|
|
@ -518,7 +500,13 @@ class Hive(Dialect):
|
||||||
"" if e.args.get("allow_null") else "NOT NULL"
|
"" if e.args.get("allow_null") else "NOT NULL"
|
||||||
),
|
),
|
||||||
exp.VarMap: var_map_sql,
|
exp.VarMap: var_map_sql,
|
||||||
exp.Create: _create_sql,
|
exp.Create: preprocess(
|
||||||
|
[
|
||||||
|
remove_unique_constraints,
|
||||||
|
ctas_with_tmp_tables_to_create_tmp_view,
|
||||||
|
move_schema_columns_to_partitioned_by,
|
||||||
|
]
|
||||||
|
),
|
||||||
exp.Quantile: rename_func("PERCENTILE"),
|
exp.Quantile: rename_func("PERCENTILE"),
|
||||||
exp.ApproxQuantile: rename_func("PERCENTILE_APPROX"),
|
exp.ApproxQuantile: rename_func("PERCENTILE_APPROX"),
|
||||||
exp.RegexpExtract: regexp_extract_sql,
|
exp.RegexpExtract: regexp_extract_sql,
|
||||||
|
|
@ -581,10 +569,6 @@ class Hive(Dialect):
|
||||||
|
|
||||||
return super()._jsonpathkey_sql(expression)
|
return super()._jsonpathkey_sql(expression)
|
||||||
|
|
||||||
def temporary_storage_provider(self, expression: exp.Create) -> exp.Create:
|
|
||||||
# Hive has no temporary storage provider (there are hive settings though)
|
|
||||||
return expression
|
|
||||||
|
|
||||||
def parameter_sql(self, expression: exp.Parameter) -> str:
|
def parameter_sql(self, expression: exp.Parameter) -> str:
|
||||||
this = self.sql(expression, "this")
|
this = self.sql(expression, "this")
|
||||||
expression_sql = self.sql(expression, "expression")
|
expression_sql = self.sql(expression, "expression")
|
||||||
|
|
|
||||||
|
|
@ -445,6 +445,7 @@ class MySQL(Dialect):
|
||||||
|
|
||||||
LOG_DEFAULTS_TO_LN = True
|
LOG_DEFAULTS_TO_LN = True
|
||||||
STRING_ALIASES = True
|
STRING_ALIASES = True
|
||||||
|
VALUES_FOLLOWED_BY_PAREN = False
|
||||||
|
|
||||||
def _parse_primary_key_part(self) -> t.Optional[exp.Expression]:
|
def _parse_primary_key_part(self) -> t.Optional[exp.Expression]:
|
||||||
this = self._parse_id_var()
|
this = self._parse_id_var()
|
||||||
|
|
|
||||||
|
|
@ -88,6 +88,7 @@ class Oracle(Dialect):
|
||||||
class Parser(parser.Parser):
|
class Parser(parser.Parser):
|
||||||
ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = False
|
ALTER_TABLE_ADD_REQUIRED_FOR_EACH_COLUMN = False
|
||||||
WINDOW_BEFORE_PAREN_TOKENS = {TokenType.OVER, TokenType.KEEP}
|
WINDOW_BEFORE_PAREN_TOKENS = {TokenType.OVER, TokenType.KEEP}
|
||||||
|
VALUES_FOLLOWED_BY_PAREN = False
|
||||||
|
|
||||||
FUNCTIONS = {
|
FUNCTIONS = {
|
||||||
**parser.Parser.FUNCTIONS,
|
**parser.Parser.FUNCTIONS,
|
||||||
|
|
|
||||||
|
|
@ -244,6 +244,8 @@ class Postgres(Dialect):
|
||||||
"@@": TokenType.DAT,
|
"@@": TokenType.DAT,
|
||||||
"@>": TokenType.AT_GT,
|
"@>": TokenType.AT_GT,
|
||||||
"<@": TokenType.LT_AT,
|
"<@": TokenType.LT_AT,
|
||||||
|
"|/": TokenType.PIPE_SLASH,
|
||||||
|
"||/": TokenType.DPIPE_SLASH,
|
||||||
"BEGIN": TokenType.COMMAND,
|
"BEGIN": TokenType.COMMAND,
|
||||||
"BEGIN TRANSACTION": TokenType.BEGIN,
|
"BEGIN TRANSACTION": TokenType.BEGIN,
|
||||||
"BIGSERIAL": TokenType.BIGSERIAL,
|
"BIGSERIAL": TokenType.BIGSERIAL,
|
||||||
|
|
|
||||||
|
|
@ -225,6 +225,8 @@ class Presto(Dialect):
|
||||||
}
|
}
|
||||||
|
|
||||||
class Parser(parser.Parser):
|
class Parser(parser.Parser):
|
||||||
|
VALUES_FOLLOWED_BY_PAREN = False
|
||||||
|
|
||||||
FUNCTIONS = {
|
FUNCTIONS = {
|
||||||
**parser.Parser.FUNCTIONS,
|
**parser.Parser.FUNCTIONS,
|
||||||
"ARBITRARY": exp.AnyValue.from_arg_list,
|
"ARBITRARY": exp.AnyValue.from_arg_list,
|
||||||
|
|
|
||||||
|
|
@ -136,11 +136,11 @@ class Redshift(Postgres):
|
||||||
refs.add(
|
refs.add(
|
||||||
(
|
(
|
||||||
this.args["from"] if i == 0 else this.args["joins"][i - 1]
|
this.args["from"] if i == 0 else this.args["joins"][i - 1]
|
||||||
).alias_or_name.lower()
|
).this.alias.lower()
|
||||||
)
|
)
|
||||||
table = join.this
|
|
||||||
|
|
||||||
if isinstance(table, exp.Table):
|
table = join.this
|
||||||
|
if isinstance(table, exp.Table) and not join.args.get("on"):
|
||||||
if table.parts[0].name.lower() in refs:
|
if table.parts[0].name.lower() in refs:
|
||||||
table.replace(table.to_column())
|
table.replace(table.to_column())
|
||||||
return this
|
return this
|
||||||
|
|
@ -158,6 +158,7 @@ class Redshift(Postgres):
|
||||||
"UNLOAD": TokenType.COMMAND,
|
"UNLOAD": TokenType.COMMAND,
|
||||||
"VARBYTE": TokenType.VARBINARY,
|
"VARBYTE": TokenType.VARBINARY,
|
||||||
}
|
}
|
||||||
|
KEYWORDS.pop("VALUES")
|
||||||
|
|
||||||
# Redshift allows # to appear as a table identifier prefix
|
# Redshift allows # to appear as a table identifier prefix
|
||||||
SINGLE_TOKENS = Postgres.Tokenizer.SINGLE_TOKENS.copy()
|
SINGLE_TOKENS = Postgres.Tokenizer.SINGLE_TOKENS.copy()
|
||||||
|
|
|
||||||
|
|
@ -477,6 +477,8 @@ class Snowflake(Dialect):
|
||||||
"PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
|
"PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
|
||||||
"TERSE PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
|
"TERSE PRIMARY KEYS": _show_parser("PRIMARY KEYS"),
|
||||||
"COLUMNS": _show_parser("COLUMNS"),
|
"COLUMNS": _show_parser("COLUMNS"),
|
||||||
|
"USERS": _show_parser("USERS"),
|
||||||
|
"TERSE USERS": _show_parser("USERS"),
|
||||||
}
|
}
|
||||||
|
|
||||||
STAGED_FILE_SINGLE_TOKENS = {
|
STAGED_FILE_SINGLE_TOKENS = {
|
||||||
|
|
|
||||||
|
|
@ -5,8 +5,14 @@ import typing as t
|
||||||
from sqlglot import exp
|
from sqlglot import exp
|
||||||
from sqlglot.dialects.dialect import rename_func
|
from sqlglot.dialects.dialect import rename_func
|
||||||
from sqlglot.dialects.hive import _parse_ignore_nulls
|
from sqlglot.dialects.hive import _parse_ignore_nulls
|
||||||
from sqlglot.dialects.spark2 import Spark2
|
from sqlglot.dialects.spark2 import Spark2, temporary_storage_provider
|
||||||
from sqlglot.helper import seq_get
|
from sqlglot.helper import seq_get
|
||||||
|
from sqlglot.transforms import (
|
||||||
|
ctas_with_tmp_tables_to_create_tmp_view,
|
||||||
|
remove_unique_constraints,
|
||||||
|
preprocess,
|
||||||
|
move_partitioned_by_to_schema_columns,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _parse_datediff(args: t.List) -> exp.Expression:
|
def _parse_datediff(args: t.List) -> exp.Expression:
|
||||||
|
|
@ -35,6 +41,15 @@ def _parse_datediff(args: t.List) -> exp.Expression:
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_partition(e: exp.Expression) -> exp.Expression:
|
||||||
|
"""Normalize the expressions in PARTITION BY (<expression>, <expression>, ...)"""
|
||||||
|
if isinstance(e, str):
|
||||||
|
return exp.to_identifier(e)
|
||||||
|
if isinstance(e, exp.Literal):
|
||||||
|
return exp.to_identifier(e.name)
|
||||||
|
return e
|
||||||
|
|
||||||
|
|
||||||
class Spark(Spark2):
|
class Spark(Spark2):
|
||||||
class Tokenizer(Spark2.Tokenizer):
|
class Tokenizer(Spark2.Tokenizer):
|
||||||
RAW_STRINGS = [
|
RAW_STRINGS = [
|
||||||
|
|
@ -72,6 +87,17 @@ class Spark(Spark2):
|
||||||
|
|
||||||
TRANSFORMS = {
|
TRANSFORMS = {
|
||||||
**Spark2.Generator.TRANSFORMS,
|
**Spark2.Generator.TRANSFORMS,
|
||||||
|
exp.Create: preprocess(
|
||||||
|
[
|
||||||
|
remove_unique_constraints,
|
||||||
|
lambda e: ctas_with_tmp_tables_to_create_tmp_view(
|
||||||
|
e, temporary_storage_provider
|
||||||
|
),
|
||||||
|
move_partitioned_by_to_schema_columns,
|
||||||
|
]
|
||||||
|
),
|
||||||
|
exp.PartitionedByProperty: lambda self,
|
||||||
|
e: f"PARTITIONED BY {self.wrap(self.expressions(sqls=[_normalize_partition(e) for e in e.this.expressions], skip_first=True))}",
|
||||||
exp.StartsWith: rename_func("STARTSWITH"),
|
exp.StartsWith: rename_func("STARTSWITH"),
|
||||||
exp.TimestampAdd: lambda self, e: self.func(
|
exp.TimestampAdd: lambda self, e: self.func(
|
||||||
"DATEADD", e.args.get("unit") or "DAY", e.expression, e.this
|
"DATEADD", e.args.get("unit") or "DAY", e.expression, e.this
|
||||||
|
|
|
||||||
|
|
@ -13,6 +13,12 @@ from sqlglot.dialects.dialect import (
|
||||||
)
|
)
|
||||||
from sqlglot.dialects.hive import Hive
|
from sqlglot.dialects.hive import Hive
|
||||||
from sqlglot.helper import seq_get
|
from sqlglot.helper import seq_get
|
||||||
|
from sqlglot.transforms import (
|
||||||
|
preprocess,
|
||||||
|
remove_unique_constraints,
|
||||||
|
ctas_with_tmp_tables_to_create_tmp_view,
|
||||||
|
move_schema_columns_to_partitioned_by,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _map_sql(self: Spark2.Generator, expression: exp.Map) -> str:
|
def _map_sql(self: Spark2.Generator, expression: exp.Map) -> str:
|
||||||
|
|
@ -95,6 +101,13 @@ def _unqualify_pivot_columns(expression: exp.Expression) -> exp.Expression:
|
||||||
return expression
|
return expression
|
||||||
|
|
||||||
|
|
||||||
|
def temporary_storage_provider(expression: exp.Expression) -> exp.Expression:
|
||||||
|
# spark2, spark, Databricks require a storage provider for temporary tables
|
||||||
|
provider = exp.FileFormatProperty(this=exp.Literal.string("parquet"))
|
||||||
|
expression.args["properties"].append("expressions", provider)
|
||||||
|
return expression
|
||||||
|
|
||||||
|
|
||||||
class Spark2(Hive):
|
class Spark2(Hive):
|
||||||
class Parser(Hive.Parser):
|
class Parser(Hive.Parser):
|
||||||
TRIM_PATTERN_FIRST = True
|
TRIM_PATTERN_FIRST = True
|
||||||
|
|
@ -121,7 +134,6 @@ class Spark2(Hive):
|
||||||
),
|
),
|
||||||
zone=seq_get(args, 1),
|
zone=seq_get(args, 1),
|
||||||
),
|
),
|
||||||
"IIF": exp.If.from_arg_list,
|
|
||||||
"INT": _parse_as_cast("int"),
|
"INT": _parse_as_cast("int"),
|
||||||
"MAP_FROM_ARRAYS": exp.Map.from_arg_list,
|
"MAP_FROM_ARRAYS": exp.Map.from_arg_list,
|
||||||
"RLIKE": exp.RegexpLike.from_arg_list,
|
"RLIKE": exp.RegexpLike.from_arg_list,
|
||||||
|
|
@ -193,6 +205,15 @@ class Spark2(Hive):
|
||||||
e: f"FROM_UTC_TIMESTAMP({self.sql(e, 'this')}, {self.sql(e, 'zone')})",
|
e: f"FROM_UTC_TIMESTAMP({self.sql(e, 'this')}, {self.sql(e, 'zone')})",
|
||||||
exp.BitwiseLeftShift: rename_func("SHIFTLEFT"),
|
exp.BitwiseLeftShift: rename_func("SHIFTLEFT"),
|
||||||
exp.BitwiseRightShift: rename_func("SHIFTRIGHT"),
|
exp.BitwiseRightShift: rename_func("SHIFTRIGHT"),
|
||||||
|
exp.Create: preprocess(
|
||||||
|
[
|
||||||
|
remove_unique_constraints,
|
||||||
|
lambda e: ctas_with_tmp_tables_to_create_tmp_view(
|
||||||
|
e, temporary_storage_provider
|
||||||
|
),
|
||||||
|
move_schema_columns_to_partitioned_by,
|
||||||
|
]
|
||||||
|
),
|
||||||
exp.DateFromParts: rename_func("MAKE_DATE"),
|
exp.DateFromParts: rename_func("MAKE_DATE"),
|
||||||
exp.DateTrunc: lambda self, e: self.func("TRUNC", e.this, e.args.get("unit")),
|
exp.DateTrunc: lambda self, e: self.func("TRUNC", e.this, e.args.get("unit")),
|
||||||
exp.DayOfMonth: rename_func("DAYOFMONTH"),
|
exp.DayOfMonth: rename_func("DAYOFMONTH"),
|
||||||
|
|
@ -251,12 +272,6 @@ class Spark2(Hive):
|
||||||
|
|
||||||
return self.func("STRUCT", *args)
|
return self.func("STRUCT", *args)
|
||||||
|
|
||||||
def temporary_storage_provider(self, expression: exp.Create) -> exp.Create:
|
|
||||||
# spark2, spark, Databricks require a storage provider for temporary tables
|
|
||||||
provider = exp.FileFormatProperty(this=exp.Literal.string("parquet"))
|
|
||||||
expression.args["properties"].append("expressions", provider)
|
|
||||||
return expression
|
|
||||||
|
|
||||||
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
|
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
|
||||||
if is_parse_json(expression.this):
|
if is_parse_json(expression.this):
|
||||||
schema = f"'{self.sql(expression, 'to')}'"
|
schema = f"'{self.sql(expression, 'to')}'"
|
||||||
|
|
|
||||||
|
|
@ -132,6 +132,7 @@ class SQLite(Dialect):
|
||||||
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
|
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
|
||||||
exp.DateAdd: _date_add_sql,
|
exp.DateAdd: _date_add_sql,
|
||||||
exp.DateStrToDate: lambda self, e: self.sql(e, "this"),
|
exp.DateStrToDate: lambda self, e: self.sql(e, "this"),
|
||||||
|
exp.If: rename_func("IIF"),
|
||||||
exp.ILike: no_ilike_sql,
|
exp.ILike: no_ilike_sql,
|
||||||
exp.JSONExtract: _json_extract_sql,
|
exp.JSONExtract: _json_extract_sql,
|
||||||
exp.JSONExtractScalar: arrow_json_extract_sql,
|
exp.JSONExtractScalar: arrow_json_extract_sql,
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,14 @@
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from sqlglot import exp, generator, parser, transforms
|
from sqlglot import exp, generator, parser, tokens, transforms
|
||||||
from sqlglot.dialects.dialect import Dialect, rename_func
|
from sqlglot.dialects.dialect import Dialect, rename_func
|
||||||
|
|
||||||
|
|
||||||
class Tableau(Dialect):
|
class Tableau(Dialect):
|
||||||
|
class Tokenizer(tokens.Tokenizer):
|
||||||
|
IDENTIFIERS = [("[", "]")]
|
||||||
|
QUOTES = ["'", '"']
|
||||||
|
|
||||||
class Generator(generator.Generator):
|
class Generator(generator.Generator):
|
||||||
JOIN_HINTS = False
|
JOIN_HINTS = False
|
||||||
TABLE_HINTS = False
|
TABLE_HINTS = False
|
||||||
|
|
|
||||||
|
|
@ -74,6 +74,7 @@ class Teradata(Dialect):
|
||||||
|
|
||||||
class Parser(parser.Parser):
|
class Parser(parser.Parser):
|
||||||
TABLESAMPLE_CSV = True
|
TABLESAMPLE_CSV = True
|
||||||
|
VALUES_FOLLOWED_BY_PAREN = False
|
||||||
|
|
||||||
CHARSET_TRANSLATORS = {
|
CHARSET_TRANSLATORS = {
|
||||||
"GRAPHIC_TO_KANJISJIS",
|
"GRAPHIC_TO_KANJISJIS",
|
||||||
|
|
|
||||||
|
|
@ -457,7 +457,6 @@ class TSQL(Dialect):
|
||||||
"FORMAT": _parse_format,
|
"FORMAT": _parse_format,
|
||||||
"GETDATE": exp.CurrentTimestamp.from_arg_list,
|
"GETDATE": exp.CurrentTimestamp.from_arg_list,
|
||||||
"HASHBYTES": _parse_hashbytes,
|
"HASHBYTES": _parse_hashbytes,
|
||||||
"IIF": exp.If.from_arg_list,
|
|
||||||
"ISNULL": exp.Coalesce.from_arg_list,
|
"ISNULL": exp.Coalesce.from_arg_list,
|
||||||
"JSON_QUERY": parser.parse_extract_json_with_path(exp.JSONExtract),
|
"JSON_QUERY": parser.parse_extract_json_with_path(exp.JSONExtract),
|
||||||
"JSON_VALUE": parser.parse_extract_json_with_path(exp.JSONExtractScalar),
|
"JSON_VALUE": parser.parse_extract_json_with_path(exp.JSONExtractScalar),
|
||||||
|
|
|
||||||
|
|
@ -1090,6 +1090,11 @@ class Create(DDL):
|
||||||
"clone": False,
|
"clone": False,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@property
|
||||||
|
def kind(self) -> t.Optional[str]:
|
||||||
|
kind = self.args.get("kind")
|
||||||
|
return kind and kind.upper()
|
||||||
|
|
||||||
|
|
||||||
# https://docs.snowflake.com/en/sql-reference/sql/create-clone
|
# https://docs.snowflake.com/en/sql-reference/sql/create-clone
|
||||||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_table_clone_statement
|
# https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_table_clone_statement
|
||||||
|
|
@ -4626,6 +4631,11 @@ class CountIf(AggFunc):
|
||||||
_sql_names = ["COUNT_IF", "COUNTIF"]
|
_sql_names = ["COUNT_IF", "COUNTIF"]
|
||||||
|
|
||||||
|
|
||||||
|
# cube root
|
||||||
|
class Cbrt(Func):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
class CurrentDate(Func):
|
class CurrentDate(Func):
|
||||||
arg_types = {"this": False}
|
arg_types = {"this": False}
|
||||||
|
|
||||||
|
|
@ -4728,7 +4738,7 @@ class Extract(Func):
|
||||||
|
|
||||||
|
|
||||||
class Timestamp(Func):
|
class Timestamp(Func):
|
||||||
arg_types = {"this": False, "expression": False}
|
arg_types = {"this": False, "expression": False, "with_tz": False}
|
||||||
|
|
||||||
|
|
||||||
class TimestampAdd(Func, TimeUnit):
|
class TimestampAdd(Func, TimeUnit):
|
||||||
|
|
@ -4833,7 +4843,7 @@ class Posexplode(Explode):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
class PosexplodeOuter(Posexplode):
|
class PosexplodeOuter(Posexplode, ExplodeOuter):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -4868,6 +4878,7 @@ class Xor(Connector, Func):
|
||||||
|
|
||||||
class If(Func):
|
class If(Func):
|
||||||
arg_types = {"this": True, "true": True, "false": False}
|
arg_types = {"this": True, "true": True, "false": False}
|
||||||
|
_sql_names = ["IF", "IIF"]
|
||||||
|
|
||||||
|
|
||||||
class Nullif(Func):
|
class Nullif(Func):
|
||||||
|
|
@ -6883,6 +6894,7 @@ def replace_tables(
|
||||||
table = to_table(
|
table = to_table(
|
||||||
new_name,
|
new_name,
|
||||||
**{k: v for k, v in node.args.items() if k not in TABLE_PARTS},
|
**{k: v for k, v in node.args.items() if k not in TABLE_PARTS},
|
||||||
|
dialect=dialect,
|
||||||
)
|
)
|
||||||
table.add_comments([original])
|
table.add_comments([original])
|
||||||
return table
|
return table
|
||||||
|
|
@ -7072,6 +7084,60 @@ def cast_unless(
|
||||||
return cast(expr, to, **opts)
|
return cast(expr, to, **opts)
|
||||||
|
|
||||||
|
|
||||||
|
def array(
|
||||||
|
*expressions: ExpOrStr, copy: bool = True, dialect: DialectType = None, **kwargs
|
||||||
|
) -> Array:
|
||||||
|
"""
|
||||||
|
Returns an array.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
>>> array(1, 'x').sql()
|
||||||
|
'ARRAY(1, x)'
|
||||||
|
|
||||||
|
Args:
|
||||||
|
expressions: the expressions to add to the array.
|
||||||
|
copy: whether or not to copy the argument expressions.
|
||||||
|
dialect: the source dialect.
|
||||||
|
kwargs: the kwargs used to instantiate the function of interest.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
An array expression.
|
||||||
|
"""
|
||||||
|
return Array(
|
||||||
|
expressions=[
|
||||||
|
maybe_parse(expression, copy=copy, dialect=dialect, **kwargs)
|
||||||
|
for expression in expressions
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def tuple_(
|
||||||
|
*expressions: ExpOrStr, copy: bool = True, dialect: DialectType = None, **kwargs
|
||||||
|
) -> Tuple:
|
||||||
|
"""
|
||||||
|
Returns an tuple.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
>>> tuple_(1, 'x').sql()
|
||||||
|
'(1, x)'
|
||||||
|
|
||||||
|
Args:
|
||||||
|
expressions: the expressions to add to the tuple.
|
||||||
|
copy: whether or not to copy the argument expressions.
|
||||||
|
dialect: the source dialect.
|
||||||
|
kwargs: the kwargs used to instantiate the function of interest.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
A tuple expression.
|
||||||
|
"""
|
||||||
|
return Tuple(
|
||||||
|
expressions=[
|
||||||
|
maybe_parse(expression, copy=copy, dialect=dialect, **kwargs)
|
||||||
|
for expression in expressions
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def true() -> Boolean:
|
def true() -> Boolean:
|
||||||
"""
|
"""
|
||||||
Returns a true Boolean expression.
|
Returns a true Boolean expression.
|
||||||
|
|
|
||||||
|
|
@ -124,6 +124,7 @@ class Generator(metaclass=_Generator):
|
||||||
exp.StabilityProperty: lambda self, e: e.name,
|
exp.StabilityProperty: lambda self, e: e.name,
|
||||||
exp.TemporaryProperty: lambda self, e: "TEMPORARY",
|
exp.TemporaryProperty: lambda self, e: "TEMPORARY",
|
||||||
exp.TitleColumnConstraint: lambda self, e: f"TITLE {self.sql(e, 'this')}",
|
exp.TitleColumnConstraint: lambda self, e: f"TITLE {self.sql(e, 'this')}",
|
||||||
|
exp.Timestamp: lambda self, e: self.func("TIMESTAMP", e.this, e.expression),
|
||||||
exp.ToTableProperty: lambda self, e: f"TO {self.sql(e.this)}",
|
exp.ToTableProperty: lambda self, e: f"TO {self.sql(e.this)}",
|
||||||
exp.TransformModelProperty: lambda self, e: self.func("TRANSFORM", *e.expressions),
|
exp.TransformModelProperty: lambda self, e: self.func("TRANSFORM", *e.expressions),
|
||||||
exp.TransientProperty: lambda self, e: "TRANSIENT",
|
exp.TransientProperty: lambda self, e: "TRANSIENT",
|
||||||
|
|
@ -3360,7 +3361,7 @@ class Generator(metaclass=_Generator):
|
||||||
return self.sql(arg)
|
return self.sql(arg)
|
||||||
|
|
||||||
cond_for_null = arg.is_(exp.null())
|
cond_for_null = arg.is_(exp.null())
|
||||||
return self.sql(exp.func("IF", cond_for_null, exp.null(), exp.Array(expressions=[arg])))
|
return self.sql(exp.func("IF", cond_for_null, exp.null(), exp.array(arg, copy=False)))
|
||||||
|
|
||||||
def tsordstotime_sql(self, expression: exp.TsOrDsToTime) -> str:
|
def tsordstotime_sql(self, expression: exp.TsOrDsToTime) -> str:
|
||||||
this = expression.this
|
this = expression.this
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ import logging
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
import typing as t
|
import typing as t
|
||||||
from collections.abc import Collection
|
from collections.abc import Collection, Set
|
||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
from copy import copy
|
from copy import copy
|
||||||
from enum import Enum
|
from enum import Enum
|
||||||
|
|
@ -496,3 +496,31 @@ DATE_UNITS = {"day", "week", "month", "quarter", "year", "year_month"}
|
||||||
|
|
||||||
def is_date_unit(expression: t.Optional[exp.Expression]) -> bool:
|
def is_date_unit(expression: t.Optional[exp.Expression]) -> bool:
|
||||||
return expression is not None and expression.name.lower() in DATE_UNITS
|
return expression is not None and expression.name.lower() in DATE_UNITS
|
||||||
|
|
||||||
|
|
||||||
|
K = t.TypeVar("K")
|
||||||
|
V = t.TypeVar("V")
|
||||||
|
|
||||||
|
|
||||||
|
class SingleValuedMapping(t.Mapping[K, V]):
|
||||||
|
"""
|
||||||
|
Mapping where all keys return the same value.
|
||||||
|
|
||||||
|
This rigamarole is meant to avoid copying keys, which was originally intended
|
||||||
|
as an optimization while qualifying columns for tables with lots of columns.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, keys: t.Collection[K], value: V):
|
||||||
|
self._keys = keys if isinstance(keys, Set) else set(keys)
|
||||||
|
self._value = value
|
||||||
|
|
||||||
|
def __getitem__(self, key: K) -> V:
|
||||||
|
if key in self._keys:
|
||||||
|
return self._value
|
||||||
|
raise KeyError(key)
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return len(self._keys)
|
||||||
|
|
||||||
|
def __iter__(self) -> t.Iterator[K]:
|
||||||
|
return iter(self._keys)
|
||||||
|
|
|
||||||
|
|
@ -153,7 +153,7 @@ def lineage(
|
||||||
raise ValueError(f"Could not find {column} in {scope.expression}")
|
raise ValueError(f"Could not find {column} in {scope.expression}")
|
||||||
|
|
||||||
for s in scope.union_scopes:
|
for s in scope.union_scopes:
|
||||||
to_node(index, scope=s, upstream=upstream)
|
to_node(index, scope=s, upstream=upstream, alias=alias)
|
||||||
|
|
||||||
return upstream
|
return upstream
|
||||||
|
|
||||||
|
|
@ -209,7 +209,11 @@ def lineage(
|
||||||
if isinstance(source, Scope):
|
if isinstance(source, Scope):
|
||||||
# The table itself came from a more specific scope. Recurse into that one using the unaliased column name.
|
# The table itself came from a more specific scope. Recurse into that one using the unaliased column name.
|
||||||
to_node(
|
to_node(
|
||||||
c.name, scope=source, scope_name=table, upstream=node, alias=aliases.get(table)
|
c.name,
|
||||||
|
scope=source,
|
||||||
|
scope_name=table,
|
||||||
|
upstream=node,
|
||||||
|
alias=aliases.get(table) or alias,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# The source is not a scope - we've reached the end of the line. At this point, if a source is not found
|
# The source is not a scope - we've reached the end of the line. At this point, if a source is not found
|
||||||
|
|
|
||||||
|
|
@ -204,7 +204,6 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
||||||
exp.TimeAdd,
|
exp.TimeAdd,
|
||||||
exp.TimeStrToTime,
|
exp.TimeStrToTime,
|
||||||
exp.TimeSub,
|
exp.TimeSub,
|
||||||
exp.Timestamp,
|
|
||||||
exp.TimestampAdd,
|
exp.TimestampAdd,
|
||||||
exp.TimestampSub,
|
exp.TimestampSub,
|
||||||
exp.UnixToTime,
|
exp.UnixToTime,
|
||||||
|
|
@ -276,6 +275,10 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
||||||
exp.Nullif: lambda self, e: self._annotate_by_args(e, "this", "expression"),
|
exp.Nullif: lambda self, e: self._annotate_by_args(e, "this", "expression"),
|
||||||
exp.Slice: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN),
|
exp.Slice: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.UNKNOWN),
|
||||||
exp.Sum: lambda self, e: self._annotate_by_args(e, "this", "expressions", promote=True),
|
exp.Sum: lambda self, e: self._annotate_by_args(e, "this", "expressions", promote=True),
|
||||||
|
exp.Timestamp: lambda self, e: self._annotate_with_type(
|
||||||
|
e,
|
||||||
|
exp.DataType.Type.TIMESTAMPTZ if e.args.get("with_tz") else exp.DataType.Type.TIMESTAMP,
|
||||||
|
),
|
||||||
exp.TryCast: lambda self, e: self._annotate_with_type(e, e.args["to"]),
|
exp.TryCast: lambda self, e: self._annotate_with_type(e, e.args["to"]),
|
||||||
exp.VarMap: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.MAP),
|
exp.VarMap: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.MAP),
|
||||||
exp.Struct: lambda self, e: self._annotate_by_args(e, "expressions", struct=True),
|
exp.Struct: lambda self, e: self._annotate_by_args(e, "expressions", struct=True),
|
||||||
|
|
|
||||||
|
|
@ -38,7 +38,12 @@ def replace_date_funcs(node: exp.Expression) -> exp.Expression:
|
||||||
if isinstance(node, exp.Date) and not node.expressions and not node.args.get("zone"):
|
if isinstance(node, exp.Date) and not node.expressions and not node.args.get("zone"):
|
||||||
return exp.cast(node.this, to=exp.DataType.Type.DATE)
|
return exp.cast(node.this, to=exp.DataType.Type.DATE)
|
||||||
if isinstance(node, exp.Timestamp) and not node.expression:
|
if isinstance(node, exp.Timestamp) and not node.expression:
|
||||||
return exp.cast(node.this, to=exp.DataType.Type.TIMESTAMP)
|
if not node.type:
|
||||||
|
from sqlglot.optimizer.annotate_types import annotate_types
|
||||||
|
|
||||||
|
node = annotate_types(node)
|
||||||
|
return exp.cast(node.this, to=node.type or exp.DataType.Type.TIMESTAMP)
|
||||||
|
|
||||||
return node
|
return node
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -76,9 +81,8 @@ def coerce_type(node: exp.Expression) -> exp.Expression:
|
||||||
def remove_redundant_casts(expression: exp.Expression) -> exp.Expression:
|
def remove_redundant_casts(expression: exp.Expression) -> exp.Expression:
|
||||||
if (
|
if (
|
||||||
isinstance(expression, exp.Cast)
|
isinstance(expression, exp.Cast)
|
||||||
and expression.to.type
|
|
||||||
and expression.this.type
|
and expression.this.type
|
||||||
and expression.to.type.this == expression.this.type.this
|
and expression.to.this == expression.this.type.this
|
||||||
):
|
):
|
||||||
return expression.this
|
return expression.this
|
||||||
return expression
|
return expression
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ import typing as t
|
||||||
from sqlglot import alias, exp
|
from sqlglot import alias, exp
|
||||||
from sqlglot.dialects.dialect import Dialect, DialectType
|
from sqlglot.dialects.dialect import Dialect, DialectType
|
||||||
from sqlglot.errors import OptimizeError
|
from sqlglot.errors import OptimizeError
|
||||||
from sqlglot.helper import seq_get
|
from sqlglot.helper import seq_get, SingleValuedMapping
|
||||||
from sqlglot.optimizer.scope import Scope, build_scope, traverse_scope, walk_in_scope
|
from sqlglot.optimizer.scope import Scope, build_scope, traverse_scope, walk_in_scope
|
||||||
from sqlglot.optimizer.simplify import simplify_parens
|
from sqlglot.optimizer.simplify import simplify_parens
|
||||||
from sqlglot.schema import Schema, ensure_schema
|
from sqlglot.schema import Schema, ensure_schema
|
||||||
|
|
@ -586,8 +586,8 @@ class Resolver:
|
||||||
def __init__(self, scope: Scope, schema: Schema, infer_schema: bool = True):
|
def __init__(self, scope: Scope, schema: Schema, infer_schema: bool = True):
|
||||||
self.scope = scope
|
self.scope = scope
|
||||||
self.schema = schema
|
self.schema = schema
|
||||||
self._source_columns: t.Optional[t.Dict[str, t.List[str]]] = None
|
self._source_columns: t.Optional[t.Dict[str, t.Sequence[str]]] = None
|
||||||
self._unambiguous_columns: t.Optional[t.Dict[str, str]] = None
|
self._unambiguous_columns: t.Optional[t.Mapping[str, str]] = None
|
||||||
self._all_columns: t.Optional[t.Set[str]] = None
|
self._all_columns: t.Optional[t.Set[str]] = None
|
||||||
self._infer_schema = infer_schema
|
self._infer_schema = infer_schema
|
||||||
|
|
||||||
|
|
@ -640,7 +640,7 @@ class Resolver:
|
||||||
}
|
}
|
||||||
return self._all_columns
|
return self._all_columns
|
||||||
|
|
||||||
def get_source_columns(self, name: str, only_visible: bool = False) -> t.List[str]:
|
def get_source_columns(self, name: str, only_visible: bool = False) -> t.Sequence[str]:
|
||||||
"""Resolve the source columns for a given source `name`."""
|
"""Resolve the source columns for a given source `name`."""
|
||||||
if name not in self.scope.sources:
|
if name not in self.scope.sources:
|
||||||
raise OptimizeError(f"Unknown table: {name}")
|
raise OptimizeError(f"Unknown table: {name}")
|
||||||
|
|
@ -662,10 +662,15 @@ class Resolver:
|
||||||
else:
|
else:
|
||||||
column_aliases = []
|
column_aliases = []
|
||||||
|
|
||||||
# If the source's columns are aliased, their aliases shadow the corresponding column names
|
if column_aliases:
|
||||||
return [alias or name for (name, alias) in itertools.zip_longest(columns, column_aliases)]
|
# If the source's columns are aliased, their aliases shadow the corresponding column names.
|
||||||
|
# This can be expensive if there are lots of columns, so only do this if column_aliases exist.
|
||||||
|
return [
|
||||||
|
alias or name for (name, alias) in itertools.zip_longest(columns, column_aliases)
|
||||||
|
]
|
||||||
|
return columns
|
||||||
|
|
||||||
def _get_all_source_columns(self) -> t.Dict[str, t.List[str]]:
|
def _get_all_source_columns(self) -> t.Dict[str, t.Sequence[str]]:
|
||||||
if self._source_columns is None:
|
if self._source_columns is None:
|
||||||
self._source_columns = {
|
self._source_columns = {
|
||||||
source_name: self.get_source_columns(source_name)
|
source_name: self.get_source_columns(source_name)
|
||||||
|
|
@ -676,8 +681,8 @@ class Resolver:
|
||||||
return self._source_columns
|
return self._source_columns
|
||||||
|
|
||||||
def _get_unambiguous_columns(
|
def _get_unambiguous_columns(
|
||||||
self, source_columns: t.Dict[str, t.List[str]]
|
self, source_columns: t.Dict[str, t.Sequence[str]]
|
||||||
) -> t.Dict[str, str]:
|
) -> t.Mapping[str, str]:
|
||||||
"""
|
"""
|
||||||
Find all the unambiguous columns in sources.
|
Find all the unambiguous columns in sources.
|
||||||
|
|
||||||
|
|
@ -693,12 +698,17 @@ class Resolver:
|
||||||
source_columns_pairs = list(source_columns.items())
|
source_columns_pairs = list(source_columns.items())
|
||||||
|
|
||||||
first_table, first_columns = source_columns_pairs[0]
|
first_table, first_columns = source_columns_pairs[0]
|
||||||
unambiguous_columns = {col: first_table for col in self._find_unique_columns(first_columns)}
|
|
||||||
|
if len(source_columns_pairs) == 1:
|
||||||
|
# Performance optimization - avoid copying first_columns if there is only one table.
|
||||||
|
return SingleValuedMapping(first_columns, first_table)
|
||||||
|
|
||||||
|
unambiguous_columns = {col: first_table for col in first_columns}
|
||||||
all_columns = set(unambiguous_columns)
|
all_columns = set(unambiguous_columns)
|
||||||
|
|
||||||
for table, columns in source_columns_pairs[1:]:
|
for table, columns in source_columns_pairs[1:]:
|
||||||
unique = self._find_unique_columns(columns)
|
unique = set(columns)
|
||||||
ambiguous = set(all_columns).intersection(unique)
|
ambiguous = all_columns.intersection(unique)
|
||||||
all_columns.update(columns)
|
all_columns.update(columns)
|
||||||
|
|
||||||
for column in ambiguous:
|
for column in ambiguous:
|
||||||
|
|
@ -707,19 +717,3 @@ class Resolver:
|
||||||
unambiguous_columns[column] = table
|
unambiguous_columns[column] = table
|
||||||
|
|
||||||
return unambiguous_columns
|
return unambiguous_columns
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _find_unique_columns(columns: t.Collection[str]) -> t.Set[str]:
|
|
||||||
"""
|
|
||||||
Find the unique columns in a list of columns.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
>>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))
|
|
||||||
['a', 'c']
|
|
||||||
|
|
||||||
This is necessary because duplicate column names are ambiguous.
|
|
||||||
"""
|
|
||||||
counts: t.Dict[str, int] = {}
|
|
||||||
for column in columns:
|
|
||||||
counts[column] = counts.get(column, 0) + 1
|
|
||||||
return {column for column, count in counts.items() if count == 1}
|
|
||||||
|
|
|
||||||
|
|
@ -29,8 +29,8 @@ def parse_var_map(args: t.List) -> exp.StarMap | exp.VarMap:
|
||||||
values.append(args[i + 1])
|
values.append(args[i + 1])
|
||||||
|
|
||||||
return exp.VarMap(
|
return exp.VarMap(
|
||||||
keys=exp.Array(expressions=keys),
|
keys=exp.array(*keys, copy=False),
|
||||||
values=exp.Array(expressions=values),
|
values=exp.array(*values, copy=False),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -638,6 +638,8 @@ class Parser(metaclass=_Parser):
|
||||||
TokenType.NOT: lambda self: self.expression(exp.Not, this=self._parse_equality()),
|
TokenType.NOT: lambda self: self.expression(exp.Not, this=self._parse_equality()),
|
||||||
TokenType.TILDA: lambda self: self.expression(exp.BitwiseNot, this=self._parse_unary()),
|
TokenType.TILDA: lambda self: self.expression(exp.BitwiseNot, this=self._parse_unary()),
|
||||||
TokenType.DASH: lambda self: self.expression(exp.Neg, this=self._parse_unary()),
|
TokenType.DASH: lambda self: self.expression(exp.Neg, this=self._parse_unary()),
|
||||||
|
TokenType.PIPE_SLASH: lambda self: self.expression(exp.Sqrt, this=self._parse_unary()),
|
||||||
|
TokenType.DPIPE_SLASH: lambda self: self.expression(exp.Cbrt, this=self._parse_unary()),
|
||||||
}
|
}
|
||||||
|
|
||||||
PRIMARY_PARSERS = {
|
PRIMARY_PARSERS = {
|
||||||
|
|
@ -1000,9 +1002,13 @@ class Parser(metaclass=_Parser):
|
||||||
MODIFIERS_ATTACHED_TO_UNION = True
|
MODIFIERS_ATTACHED_TO_UNION = True
|
||||||
UNION_MODIFIERS = {"order", "limit", "offset"}
|
UNION_MODIFIERS = {"order", "limit", "offset"}
|
||||||
|
|
||||||
# parses no parenthesis if statements as commands
|
# Parses no parenthesis if statements as commands
|
||||||
NO_PAREN_IF_COMMANDS = True
|
NO_PAREN_IF_COMMANDS = True
|
||||||
|
|
||||||
|
# Whether or not a VALUES keyword needs to be followed by '(' to form a VALUES clause.
|
||||||
|
# If this is True and '(' is not found, the keyword will be treated as an identifier
|
||||||
|
VALUES_FOLLOWED_BY_PAREN = True
|
||||||
|
|
||||||
__slots__ = (
|
__slots__ = (
|
||||||
"error_level",
|
"error_level",
|
||||||
"error_message_context",
|
"error_message_context",
|
||||||
|
|
@ -2058,7 +2064,7 @@ class Parser(metaclass=_Parser):
|
||||||
partition=self._parse_partition(),
|
partition=self._parse_partition(),
|
||||||
where=self._match_pair(TokenType.REPLACE, TokenType.WHERE)
|
where=self._match_pair(TokenType.REPLACE, TokenType.WHERE)
|
||||||
and self._parse_conjunction(),
|
and self._parse_conjunction(),
|
||||||
expression=self._parse_ddl_select(),
|
expression=self._parse_derived_table_values() or self._parse_ddl_select(),
|
||||||
conflict=self._parse_on_conflict(),
|
conflict=self._parse_on_conflict(),
|
||||||
returning=returning or self._parse_returning(),
|
returning=returning or self._parse_returning(),
|
||||||
overwrite=overwrite,
|
overwrite=overwrite,
|
||||||
|
|
@ -2267,8 +2273,7 @@ class Parser(metaclass=_Parser):
|
||||||
self._match_r_paren()
|
self._match_r_paren()
|
||||||
return self.expression(exp.Tuple, expressions=expressions)
|
return self.expression(exp.Tuple, expressions=expressions)
|
||||||
|
|
||||||
# In presto we can have VALUES 1, 2 which results in 1 column & 2 rows.
|
# In some dialects we can have VALUES 1, 2 which results in 1 column & 2 rows.
|
||||||
# https://prestodb.io/docs/current/sql/values.html
|
|
||||||
return self.expression(exp.Tuple, expressions=[self._parse_expression()])
|
return self.expression(exp.Tuple, expressions=[self._parse_expression()])
|
||||||
|
|
||||||
def _parse_projections(self) -> t.List[exp.Expression]:
|
def _parse_projections(self) -> t.List[exp.Expression]:
|
||||||
|
|
@ -2367,12 +2372,8 @@ class Parser(metaclass=_Parser):
|
||||||
# We return early here so that the UNION isn't attached to the subquery by the
|
# We return early here so that the UNION isn't attached to the subquery by the
|
||||||
# following call to _parse_set_operations, but instead becomes the parent node
|
# following call to _parse_set_operations, but instead becomes the parent node
|
||||||
return self._parse_subquery(this, parse_alias=parse_subquery_alias)
|
return self._parse_subquery(this, parse_alias=parse_subquery_alias)
|
||||||
elif self._match(TokenType.VALUES):
|
elif self._match(TokenType.VALUES, advance=False):
|
||||||
this = self.expression(
|
this = self._parse_derived_table_values()
|
||||||
exp.Values,
|
|
||||||
expressions=self._parse_csv(self._parse_value),
|
|
||||||
alias=self._parse_table_alias(),
|
|
||||||
)
|
|
||||||
elif from_:
|
elif from_:
|
||||||
this = exp.select("*").from_(from_.this, copy=False)
|
this = exp.select("*").from_(from_.this, copy=False)
|
||||||
else:
|
else:
|
||||||
|
|
@ -2969,7 +2970,7 @@ class Parser(metaclass=_Parser):
|
||||||
|
|
||||||
def _parse_derived_table_values(self) -> t.Optional[exp.Values]:
|
def _parse_derived_table_values(self) -> t.Optional[exp.Values]:
|
||||||
is_derived = self._match_pair(TokenType.L_PAREN, TokenType.VALUES)
|
is_derived = self._match_pair(TokenType.L_PAREN, TokenType.VALUES)
|
||||||
if not is_derived and not self._match(TokenType.VALUES):
|
if not is_derived and not self._match_text_seq("VALUES"):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
expressions = self._parse_csv(self._parse_value)
|
expressions = self._parse_csv(self._parse_value)
|
||||||
|
|
@ -3655,8 +3656,15 @@ class Parser(metaclass=_Parser):
|
||||||
def _parse_type(self, parse_interval: bool = True) -> t.Optional[exp.Expression]:
|
def _parse_type(self, parse_interval: bool = True) -> t.Optional[exp.Expression]:
|
||||||
interval = parse_interval and self._parse_interval()
|
interval = parse_interval and self._parse_interval()
|
||||||
if interval:
|
if interval:
|
||||||
# Convert INTERVAL 'val_1' unit_1 ... 'val_n' unit_n into a sum of intervals
|
# Convert INTERVAL 'val_1' unit_1 [+] ... [+] 'val_n' unit_n into a sum of intervals
|
||||||
while self._match_set((TokenType.STRING, TokenType.NUMBER), advance=False):
|
while True:
|
||||||
|
index = self._index
|
||||||
|
self._match(TokenType.PLUS)
|
||||||
|
|
||||||
|
if not self._match_set((TokenType.STRING, TokenType.NUMBER), advance=False):
|
||||||
|
self._retreat(index)
|
||||||
|
break
|
||||||
|
|
||||||
interval = self.expression( # type: ignore
|
interval = self.expression( # type: ignore
|
||||||
exp.Add, this=interval, expression=self._parse_interval(match_interval=False)
|
exp.Add, this=interval, expression=self._parse_interval(match_interval=False)
|
||||||
)
|
)
|
||||||
|
|
@ -3872,9 +3880,15 @@ class Parser(metaclass=_Parser):
|
||||||
|
|
||||||
def _parse_column_reference(self) -> t.Optional[exp.Expression]:
|
def _parse_column_reference(self) -> t.Optional[exp.Expression]:
|
||||||
this = self._parse_field()
|
this = self._parse_field()
|
||||||
if isinstance(this, exp.Identifier):
|
if (
|
||||||
this = self.expression(exp.Column, this=this)
|
not this
|
||||||
return this
|
and self._match(TokenType.VALUES, advance=False)
|
||||||
|
and self.VALUES_FOLLOWED_BY_PAREN
|
||||||
|
and (not self._next or self._next.token_type != TokenType.L_PAREN)
|
||||||
|
):
|
||||||
|
this = self._parse_id_var()
|
||||||
|
|
||||||
|
return self.expression(exp.Column, this=this) if isinstance(this, exp.Identifier) else this
|
||||||
|
|
||||||
def _parse_column_ops(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
|
def _parse_column_ops(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
|
||||||
this = self._parse_bracket(this)
|
this = self._parse_bracket(this)
|
||||||
|
|
@ -5511,7 +5525,7 @@ class Parser(metaclass=_Parser):
|
||||||
then = self.expression(
|
then = self.expression(
|
||||||
exp.Insert,
|
exp.Insert,
|
||||||
this=self._parse_value(),
|
this=self._parse_value(),
|
||||||
expression=self._match(TokenType.VALUES) and self._parse_value(),
|
expression=self._match_text_seq("VALUES") and self._parse_value(),
|
||||||
)
|
)
|
||||||
elif self._match(TokenType.UPDATE):
|
elif self._match(TokenType.UPDATE):
|
||||||
expressions = self._parse_star()
|
expressions = self._parse_star()
|
||||||
|
|
|
||||||
|
|
@ -49,7 +49,7 @@ class Schema(abc.ABC):
|
||||||
only_visible: bool = False,
|
only_visible: bool = False,
|
||||||
dialect: DialectType = None,
|
dialect: DialectType = None,
|
||||||
normalize: t.Optional[bool] = None,
|
normalize: t.Optional[bool] = None,
|
||||||
) -> t.List[str]:
|
) -> t.Sequence[str]:
|
||||||
"""
|
"""
|
||||||
Get the column names for a table.
|
Get the column names for a table.
|
||||||
|
|
||||||
|
|
@ -60,7 +60,7 @@ class Schema(abc.ABC):
|
||||||
normalize: whether to normalize identifiers according to the dialect of interest.
|
normalize: whether to normalize identifiers according to the dialect of interest.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
The list of column names.
|
The sequence of column names.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@abc.abstractmethod
|
@abc.abstractmethod
|
||||||
|
|
|
||||||
|
|
@ -57,6 +57,8 @@ class TokenType(AutoName):
|
||||||
AMP = auto()
|
AMP = auto()
|
||||||
DPIPE = auto()
|
DPIPE = auto()
|
||||||
PIPE = auto()
|
PIPE = auto()
|
||||||
|
PIPE_SLASH = auto()
|
||||||
|
DPIPE_SLASH = auto()
|
||||||
CARET = auto()
|
CARET = auto()
|
||||||
TILDA = auto()
|
TILDA = auto()
|
||||||
ARROW = auto()
|
ARROW = auto()
|
||||||
|
|
|
||||||
|
|
@ -213,6 +213,19 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
|
||||||
is_posexplode = isinstance(explode, exp.Posexplode)
|
is_posexplode = isinstance(explode, exp.Posexplode)
|
||||||
explode_arg = explode.this
|
explode_arg = explode.this
|
||||||
|
|
||||||
|
if isinstance(explode, exp.ExplodeOuter):
|
||||||
|
bracket = explode_arg[0]
|
||||||
|
bracket.set("safe", True)
|
||||||
|
bracket.set("offset", True)
|
||||||
|
explode_arg = exp.func(
|
||||||
|
"IF",
|
||||||
|
exp.func(
|
||||||
|
"ARRAY_SIZE", exp.func("COALESCE", explode_arg, exp.Array())
|
||||||
|
).eq(0),
|
||||||
|
exp.array(bracket, copy=False),
|
||||||
|
explode_arg,
|
||||||
|
)
|
||||||
|
|
||||||
# This ensures that we won't use [POS]EXPLODE's argument as a new selection
|
# This ensures that we won't use [POS]EXPLODE's argument as a new selection
|
||||||
if isinstance(explode_arg, exp.Column):
|
if isinstance(explode_arg, exp.Column):
|
||||||
taken_select_names.add(explode_arg.output_name)
|
taken_select_names.add(explode_arg.output_name)
|
||||||
|
|
@ -466,6 +479,87 @@ def unqualify_columns(expression: exp.Expression) -> exp.Expression:
|
||||||
return expression
|
return expression
|
||||||
|
|
||||||
|
|
||||||
|
def remove_unique_constraints(expression: exp.Expression) -> exp.Expression:
|
||||||
|
assert isinstance(expression, exp.Create)
|
||||||
|
for constraint in expression.find_all(exp.UniqueColumnConstraint):
|
||||||
|
if constraint.parent:
|
||||||
|
constraint.parent.pop()
|
||||||
|
|
||||||
|
return expression
|
||||||
|
|
||||||
|
|
||||||
|
def ctas_with_tmp_tables_to_create_tmp_view(
|
||||||
|
expression: exp.Expression,
|
||||||
|
tmp_storage_provider: t.Callable[[exp.Expression], exp.Expression] = lambda e: e,
|
||||||
|
) -> exp.Expression:
|
||||||
|
assert isinstance(expression, exp.Create)
|
||||||
|
properties = expression.args.get("properties")
|
||||||
|
temporary = any(
|
||||||
|
isinstance(prop, exp.TemporaryProperty)
|
||||||
|
for prop in (properties.expressions if properties else [])
|
||||||
|
)
|
||||||
|
|
||||||
|
# CTAS with temp tables map to CREATE TEMPORARY VIEW
|
||||||
|
if expression.kind == "TABLE" and temporary:
|
||||||
|
if expression.expression:
|
||||||
|
return exp.Create(
|
||||||
|
kind="TEMPORARY VIEW",
|
||||||
|
this=expression.this,
|
||||||
|
expression=expression.expression,
|
||||||
|
)
|
||||||
|
return tmp_storage_provider(expression)
|
||||||
|
|
||||||
|
return expression
|
||||||
|
|
||||||
|
|
||||||
|
def move_schema_columns_to_partitioned_by(expression: exp.Expression) -> exp.Expression:
|
||||||
|
"""
|
||||||
|
In Hive, the PARTITIONED BY property acts as an extension of a table's schema. When the
|
||||||
|
PARTITIONED BY value is an array of column names, they are transformed into a schema.
|
||||||
|
The corresponding columns are removed from the create statement.
|
||||||
|
"""
|
||||||
|
assert isinstance(expression, exp.Create)
|
||||||
|
has_schema = isinstance(expression.this, exp.Schema)
|
||||||
|
is_partitionable = expression.kind in {"TABLE", "VIEW"}
|
||||||
|
|
||||||
|
if has_schema and is_partitionable:
|
||||||
|
prop = expression.find(exp.PartitionedByProperty)
|
||||||
|
if prop and prop.this and not isinstance(prop.this, exp.Schema):
|
||||||
|
schema = expression.this
|
||||||
|
columns = {v.name.upper() for v in prop.this.expressions}
|
||||||
|
partitions = [col for col in schema.expressions if col.name.upper() in columns]
|
||||||
|
schema.set("expressions", [e for e in schema.expressions if e not in partitions])
|
||||||
|
prop.replace(exp.PartitionedByProperty(this=exp.Schema(expressions=partitions)))
|
||||||
|
expression.set("this", schema)
|
||||||
|
|
||||||
|
return expression
|
||||||
|
|
||||||
|
|
||||||
|
def move_partitioned_by_to_schema_columns(expression: exp.Expression) -> exp.Expression:
|
||||||
|
"""
|
||||||
|
Spark 3 supports both "HIVEFORMAT" and "DATASOURCE" formats for CREATE TABLE.
|
||||||
|
|
||||||
|
Currently, SQLGlot uses the DATASOURCE format for Spark 3.
|
||||||
|
"""
|
||||||
|
assert isinstance(expression, exp.Create)
|
||||||
|
prop = expression.find(exp.PartitionedByProperty)
|
||||||
|
if (
|
||||||
|
prop
|
||||||
|
and prop.this
|
||||||
|
and isinstance(prop.this, exp.Schema)
|
||||||
|
and all(isinstance(e, exp.ColumnDef) and e.args.get("kind") for e in prop.this.expressions)
|
||||||
|
):
|
||||||
|
prop_this = exp.Tuple(
|
||||||
|
expressions=[exp.to_identifier(e.this) for e in prop.this.expressions]
|
||||||
|
)
|
||||||
|
schema = expression.this
|
||||||
|
for e in prop.this.expressions:
|
||||||
|
schema.append("expressions", e)
|
||||||
|
prop.set("this", prop_this)
|
||||||
|
|
||||||
|
return expression
|
||||||
|
|
||||||
|
|
||||||
def preprocess(
|
def preprocess(
|
||||||
transforms: t.List[t.Callable[[exp.Expression], exp.Expression]],
|
transforms: t.List[t.Callable[[exp.Expression], exp.Expression]],
|
||||||
) -> t.Callable[[Generator, exp.Expression], str]:
|
) -> t.Callable[[Generator, exp.Expression], str]:
|
||||||
|
|
|
||||||
|
|
@ -18,78 +18,6 @@ class TestBigQuery(Validator):
|
||||||
maxDiff = None
|
maxDiff = None
|
||||||
|
|
||||||
def test_bigquery(self):
|
def test_bigquery(self):
|
||||||
self.validate_identity("ARRAY_AGG(x IGNORE NULLS LIMIT 1)")
|
|
||||||
self.validate_identity("ARRAY_AGG(x IGNORE NULLS ORDER BY x LIMIT 1)")
|
|
||||||
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY x LIMIT 1)")
|
|
||||||
self.validate_identity("ARRAY_AGG(x IGNORE NULLS)")
|
|
||||||
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS HAVING MAX x ORDER BY x LIMIT 1)")
|
|
||||||
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT SUM(x IGNORE NULLS) AS x",
|
|
||||||
read={
|
|
||||||
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
|
|
||||||
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
|
|
||||||
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
|
|
||||||
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
|
|
||||||
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
|
|
||||||
},
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
|
|
||||||
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
|
|
||||||
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
|
|
||||||
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
|
|
||||||
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT SUM(x RESPECT NULLS) AS x",
|
|
||||||
read={
|
|
||||||
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
|
|
||||||
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
|
|
||||||
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
|
|
||||||
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
|
|
||||||
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
|
|
||||||
},
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
|
|
||||||
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
|
|
||||||
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
|
|
||||||
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
|
|
||||||
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
|
|
||||||
"duckdb": "SELECT QUANTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
|
|
||||||
"spark": "SELECT PERCENTILE_CONT(x, 0.5) RESPECT NULLS OVER ()",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
|
|
||||||
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 10) AS x",
|
|
||||||
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 10) IGNORE NULLS AS x",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
|
|
||||||
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 1, 10) AS x",
|
|
||||||
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 1, 10) IGNORE NULLS AS x",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_identity("SELECT COUNT(x RESPECT NULLS)")
|
|
||||||
self.validate_identity("SELECT LAST_VALUE(x IGNORE NULLS) OVER y AS x")
|
|
||||||
|
|
||||||
self.validate_identity(
|
|
||||||
"create or replace view test (tenant_id OPTIONS(description='Test description on table creation')) select 1 as tenant_id, 1 as customer_id;",
|
|
||||||
"CREATE OR REPLACE VIEW test (tenant_id OPTIONS (description='Test description on table creation')) AS SELECT 1 AS tenant_id, 1 AS customer_id",
|
|
||||||
)
|
|
||||||
|
|
||||||
with self.assertLogs(helper_logger) as cm:
|
with self.assertLogs(helper_logger) as cm:
|
||||||
statements = parse(
|
statements = parse(
|
||||||
"""
|
"""
|
||||||
|
|
@ -131,19 +59,12 @@ class TestBigQuery(Validator):
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"a[0]",
|
"a[0]",
|
||||||
read={
|
read={
|
||||||
|
"bigquery": "a[0]",
|
||||||
"duckdb": "a[1]",
|
"duckdb": "a[1]",
|
||||||
"presto": "a[1]",
|
"presto": "a[1]",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
self.validate_identity(
|
|
||||||
"select array_contains([1, 2, 3], 1)",
|
|
||||||
"SELECT EXISTS(SELECT 1 FROM UNNEST([1, 2, 3]) AS _col WHERE _col = 1)",
|
|
||||||
)
|
|
||||||
self.validate_identity("CREATE SCHEMA x DEFAULT COLLATE 'en'")
|
|
||||||
self.validate_identity("CREATE TABLE x (y INT64) DEFAULT COLLATE 'en'")
|
|
||||||
self.validate_identity("PARSE_JSON('{}', wide_number_mode => 'exact')")
|
|
||||||
|
|
||||||
with self.assertRaises(TokenError):
|
with self.assertRaises(TokenError):
|
||||||
transpile("'\\'", read="bigquery")
|
transpile("'\\'", read="bigquery")
|
||||||
|
|
||||||
|
|
@ -179,6 +100,16 @@ class TestBigQuery(Validator):
|
||||||
)
|
)
|
||||||
assert "'END FOR'" in cm.output[0]
|
assert "'END FOR'" in cm.output[0]
|
||||||
|
|
||||||
|
self.validate_identity("CREATE SCHEMA x DEFAULT COLLATE 'en'")
|
||||||
|
self.validate_identity("CREATE TABLE x (y INT64) DEFAULT COLLATE 'en'")
|
||||||
|
self.validate_identity("PARSE_JSON('{}', wide_number_mode => 'exact')")
|
||||||
|
self.validate_identity("FOO(values)")
|
||||||
|
self.validate_identity("STRUCT(values AS value)")
|
||||||
|
self.validate_identity("ARRAY_AGG(x IGNORE NULLS LIMIT 1)")
|
||||||
|
self.validate_identity("ARRAY_AGG(x IGNORE NULLS ORDER BY x LIMIT 1)")
|
||||||
|
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY x LIMIT 1)")
|
||||||
|
self.validate_identity("ARRAY_AGG(x IGNORE NULLS)")
|
||||||
|
self.validate_identity("ARRAY_AGG(DISTINCT x IGNORE NULLS HAVING MAX x ORDER BY x LIMIT 1)")
|
||||||
self.validate_identity("SELECT * FROM dataset.my_table TABLESAMPLE SYSTEM (10 PERCENT)")
|
self.validate_identity("SELECT * FROM dataset.my_table TABLESAMPLE SYSTEM (10 PERCENT)")
|
||||||
self.validate_identity("TIME('2008-12-25 15:30:00+08')")
|
self.validate_identity("TIME('2008-12-25 15:30:00+08')")
|
||||||
self.validate_identity("TIME('2008-12-25 15:30:00+08', 'America/Los_Angeles')")
|
self.validate_identity("TIME('2008-12-25 15:30:00+08', 'America/Los_Angeles')")
|
||||||
|
|
@ -237,6 +168,13 @@ class TestBigQuery(Validator):
|
||||||
self.validate_identity("SELECT TIMESTAMP_SECONDS(2) AS t")
|
self.validate_identity("SELECT TIMESTAMP_SECONDS(2) AS t")
|
||||||
self.validate_identity("SELECT TIMESTAMP_MILLIS(2) AS t")
|
self.validate_identity("SELECT TIMESTAMP_MILLIS(2) AS t")
|
||||||
self.validate_identity("""SELECT JSON_EXTRACT_SCALAR('{"a": 5}', '$.a')""")
|
self.validate_identity("""SELECT JSON_EXTRACT_SCALAR('{"a": 5}', '$.a')""")
|
||||||
|
self.validate_identity("UPDATE x SET y = NULL")
|
||||||
|
self.validate_identity("LOG(n, b)")
|
||||||
|
self.validate_identity("SELECT COUNT(x RESPECT NULLS)")
|
||||||
|
self.validate_identity("SELECT LAST_VALUE(x IGNORE NULLS) OVER y AS x")
|
||||||
|
self.validate_identity(
|
||||||
|
"SELECT * FROM test QUALIFY a IS DISTINCT FROM b WINDOW c AS (PARTITION BY d)"
|
||||||
|
)
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"FOR record IN (SELECT word, word_count FROM bigquery-public-data.samples.shakespeare LIMIT 5) DO SELECT record.word, record.word_count"
|
"FOR record IN (SELECT word, word_count FROM bigquery-public-data.samples.shakespeare LIMIT 5) DO SELECT record.word, record.word_count"
|
||||||
)
|
)
|
||||||
|
|
@ -264,6 +202,14 @@ class TestBigQuery(Validator):
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"""SELECT JSON_EXTRACT_SCALAR('5')""", """SELECT JSON_EXTRACT_SCALAR('5', '$')"""
|
"""SELECT JSON_EXTRACT_SCALAR('5')""", """SELECT JSON_EXTRACT_SCALAR('5', '$')"""
|
||||||
)
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"select array_contains([1, 2, 3], 1)",
|
||||||
|
"SELECT EXISTS(SELECT 1 FROM UNNEST([1, 2, 3]) AS _col WHERE _col = 1)",
|
||||||
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"create or replace view test (tenant_id OPTIONS(description='Test description on table creation')) select 1 as tenant_id, 1 as customer_id;",
|
||||||
|
"CREATE OR REPLACE VIEW test (tenant_id OPTIONS (description='Test description on table creation')) AS SELECT 1 AS tenant_id, 1 AS customer_id",
|
||||||
|
)
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"SELECT SPLIT(foo)",
|
"SELECT SPLIT(foo)",
|
||||||
"SELECT SPLIT(foo, ',')",
|
"SELECT SPLIT(foo, ',')",
|
||||||
|
|
@ -312,7 +258,81 @@ class TestBigQuery(Validator):
|
||||||
"SELECT * FROM UNNEST(x) WITH OFFSET EXCEPT DISTINCT SELECT * FROM UNNEST(y) WITH OFFSET",
|
"SELECT * FROM UNNEST(x) WITH OFFSET EXCEPT DISTINCT SELECT * FROM UNNEST(y) WITH OFFSET",
|
||||||
"SELECT * FROM UNNEST(x) WITH OFFSET AS offset EXCEPT DISTINCT SELECT * FROM UNNEST(y) WITH OFFSET AS offset",
|
"SELECT * FROM UNNEST(x) WITH OFFSET AS offset EXCEPT DISTINCT SELECT * FROM UNNEST(y) WITH OFFSET AS offset",
|
||||||
)
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) d, COUNT(*) e FOR c IN ('x', 'y'))",
|
||||||
|
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) AS d, COUNT(*) AS e FOR c IN ('x', 'y'))",
|
||||||
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
r"REGEXP_EXTRACT(svc_plugin_output, r'\\\((.*)')",
|
||||||
|
r"REGEXP_EXTRACT(svc_plugin_output, '\\\\\\((.*)')",
|
||||||
|
)
|
||||||
|
|
||||||
|
self.validate_all(
|
||||||
|
"TIMESTAMP(x)",
|
||||||
|
write={
|
||||||
|
"bigquery": "TIMESTAMP(x)",
|
||||||
|
"duckdb": "CAST(x AS TIMESTAMPTZ)",
|
||||||
|
"presto": "CAST(x AS TIMESTAMP WITH TIME ZONE)",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT SUM(x IGNORE NULLS) AS x",
|
||||||
|
read={
|
||||||
|
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
|
||||||
|
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
|
||||||
|
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
|
||||||
|
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
|
||||||
|
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
|
||||||
|
},
|
||||||
|
write={
|
||||||
|
"bigquery": "SELECT SUM(x IGNORE NULLS) AS x",
|
||||||
|
"duckdb": "SELECT SUM(x IGNORE NULLS) AS x",
|
||||||
|
"postgres": "SELECT SUM(x) IGNORE NULLS AS x",
|
||||||
|
"spark": "SELECT SUM(x) IGNORE NULLS AS x",
|
||||||
|
"snowflake": "SELECT SUM(x) IGNORE NULLS AS x",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT SUM(x RESPECT NULLS) AS x",
|
||||||
|
read={
|
||||||
|
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
|
||||||
|
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
|
||||||
|
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
|
||||||
|
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
|
||||||
|
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
|
||||||
|
},
|
||||||
|
write={
|
||||||
|
"bigquery": "SELECT SUM(x RESPECT NULLS) AS x",
|
||||||
|
"duckdb": "SELECT SUM(x RESPECT NULLS) AS x",
|
||||||
|
"postgres": "SELECT SUM(x) RESPECT NULLS AS x",
|
||||||
|
"spark": "SELECT SUM(x) RESPECT NULLS AS x",
|
||||||
|
"snowflake": "SELECT SUM(x) RESPECT NULLS AS x",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
|
||||||
|
write={
|
||||||
|
"bigquery": "SELECT PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
|
||||||
|
"duckdb": "SELECT QUANTILE_CONT(x, 0.5 RESPECT NULLS) OVER ()",
|
||||||
|
"spark": "SELECT PERCENTILE_CONT(x, 0.5) RESPECT NULLS OVER ()",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
|
||||||
|
write={
|
||||||
|
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 10) AS x",
|
||||||
|
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 10) AS x",
|
||||||
|
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 10) IGNORE NULLS AS x",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
|
||||||
|
write={
|
||||||
|
"bigquery": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a, b DESC LIMIT 1, 10) AS x",
|
||||||
|
"duckdb": "SELECT ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY a NULLS FIRST, b DESC LIMIT 1, 10) AS x",
|
||||||
|
"spark": "SELECT COLLECT_LIST(DISTINCT x ORDER BY a, b DESC LIMIT 1, 10) IGNORE NULLS AS x",
|
||||||
|
},
|
||||||
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT * FROM Produce UNPIVOT((first_half_sales, second_half_sales) FOR semesters IN ((Q1, Q2) AS 'semester_1', (Q3, Q4) AS 'semester_2'))",
|
"SELECT * FROM Produce UNPIVOT((first_half_sales, second_half_sales) FOR semesters IN ((Q1, Q2) AS 'semester_1', (Q3, Q4) AS 'semester_2'))",
|
||||||
read={
|
read={
|
||||||
|
|
@ -464,7 +484,6 @@ class TestBigQuery(Validator):
|
||||||
"duckdb": "SELECT * FROM t WHERE EXISTS(SELECT * FROM UNNEST(nums) AS _t(x) WHERE x > 1)",
|
"duckdb": "SELECT * FROM t WHERE EXISTS(SELECT * FROM UNNEST(nums) AS _t(x) WHERE x > 1)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_identity("UPDATE x SET y = NULL")
|
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"NULL",
|
"NULL",
|
||||||
read={
|
read={
|
||||||
|
|
@ -620,6 +639,14 @@ class TestBigQuery(Validator):
|
||||||
"spark": "WITH cte AS (SELECT ARRAY(1, 2, 3) AS arr) SELECT EXPLODE(arr) FROM cte"
|
"spark": "WITH cte AS (SELECT ARRAY(1, 2, 3) AS arr) SELECT EXPLODE(arr) FROM cte"
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT IF(pos = pos_2, col, NULL) AS col FROM UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], []))) - 1)) AS pos CROSS JOIN UNNEST(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) AS col WITH OFFSET AS pos_2 WHERE pos = pos_2 OR (pos > (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1) AND pos_2 = (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1))",
|
||||||
|
read={"spark": "select explode_outer([])"},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT IF(pos = pos_2, col, NULL) AS col, IF(pos = pos_2, pos_2, NULL) AS pos_2 FROM UNNEST(GENERATE_ARRAY(0, GREATEST(ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], []))) - 1)) AS pos CROSS JOIN UNNEST(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) AS col WITH OFFSET AS pos_2 WHERE pos = pos_2 OR (pos > (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1) AND pos_2 = (ARRAY_LENGTH(IF(ARRAY_LENGTH(COALESCE([], [])) = 0, [[][SAFE_ORDINAL(0)]], [])) - 1))",
|
||||||
|
read={"spark": "select posexplode_outer([])"},
|
||||||
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT AS STRUCT ARRAY(SELECT AS STRUCT b FROM x) AS y FROM z",
|
"SELECT AS STRUCT ARRAY(SELECT AS STRUCT b FROM x) AS y FROM z",
|
||||||
write={
|
write={
|
||||||
|
|
@ -660,10 +687,6 @@ class TestBigQuery(Validator):
|
||||||
"bigquery": "SELECT ARRAY(SELECT AS STRUCT 1 AS a, 2 AS b)",
|
"bigquery": "SELECT ARRAY(SELECT AS STRUCT 1 AS a, 2 AS b)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_identity(
|
|
||||||
r"REGEXP_EXTRACT(svc_plugin_output, r'\\\((.*)')",
|
|
||||||
r"REGEXP_EXTRACT(svc_plugin_output, '\\\\\\((.*)')",
|
|
||||||
)
|
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"REGEXP_CONTAINS('foo', '.*')",
|
"REGEXP_CONTAINS('foo', '.*')",
|
||||||
read={
|
read={
|
||||||
|
|
@ -986,9 +1009,6 @@ class TestBigQuery(Validator):
|
||||||
"postgres": "CURRENT_DATE AT TIME ZONE 'UTC'",
|
"postgres": "CURRENT_DATE AT TIME ZONE 'UTC'",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_identity(
|
|
||||||
"SELECT * FROM test QUALIFY a IS DISTINCT FROM b WINDOW c AS (PARTITION BY d)"
|
|
||||||
)
|
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT a FROM test WHERE a = 1 GROUP BY a HAVING a = 2 QUALIFY z ORDER BY a LIMIT 10",
|
"SELECT a FROM test WHERE a = 1 GROUP BY a HAVING a = 2 QUALIFY z ORDER BY a LIMIT 10",
|
||||||
write={
|
write={
|
||||||
|
|
@ -997,45 +1017,20 @@ class TestBigQuery(Validator):
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT cola, colb FROM (VALUES (1, 'test')) AS tab(cola, colb)",
|
"SELECT cola, colb FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb)])",
|
||||||
write={
|
read={
|
||||||
"spark": "SELECT cola, colb FROM VALUES (1, 'test') AS tab(cola, colb)",
|
|
||||||
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb)])",
|
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb)])",
|
||||||
"snowflake": "SELECT cola, colb FROM (VALUES (1, 'test')) AS tab(cola, colb)",
|
"snowflake": "SELECT cola, colb FROM (VALUES (1, 'test')) AS tab(cola, colb)",
|
||||||
},
|
"spark": "SELECT cola, colb FROM VALUES (1, 'test') AS tab(cola, colb)",
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT cola, colb FROM (VALUES (1, 'test')) AS tab",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS _c0, 'test' AS _c1)])",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT cola, colb FROM (VALUES (1, 'test'))",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS _c0, 'test' AS _c1)])",
|
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT * FROM UNNEST([STRUCT(1 AS id)]) CROSS JOIN UNNEST([STRUCT(1 AS id)])",
|
"SELECT * FROM UNNEST([STRUCT(1 AS id)]) CROSS JOIN UNNEST([STRUCT(1 AS id)])",
|
||||||
read={
|
read={
|
||||||
|
"bigquery": "SELECT * FROM UNNEST([STRUCT(1 AS id)]) CROSS JOIN UNNEST([STRUCT(1 AS id)])",
|
||||||
"postgres": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
|
"postgres": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
|
||||||
"SELECT cola, colb, colc FROM (VALUES (1, 'test', NULL)) AS tab(cola, colb, colc)",
|
|
||||||
write={
|
|
||||||
"spark": "SELECT cola, colb, colc FROM VALUES (1, 'test', NULL) AS tab(cola, colb, colc)",
|
|
||||||
"bigquery": "SELECT cola, colb, colc FROM UNNEST([STRUCT(1 AS cola, 'test' AS colb, NULL AS colc)])",
|
|
||||||
"snowflake": "SELECT cola, colb, colc FROM (VALUES (1, 'test', NULL)) AS tab(cola, colb, colc)",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) d, COUNT(*) e FOR c IN ('x', 'y'))",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT * FROM (SELECT a, b, c FROM test) PIVOT(SUM(b) AS d, COUNT(*) AS e FOR c IN ('x', 'y'))",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
|
"SELECT REGEXP_EXTRACT(abc, 'pattern(group)') FROM table",
|
||||||
write={
|
write={
|
||||||
|
|
@ -1091,8 +1086,6 @@ WHERE
|
||||||
pretty=True,
|
pretty=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
self.validate_identity("LOG(n, b)")
|
|
||||||
|
|
||||||
def test_user_defined_functions(self):
|
def test_user_defined_functions(self):
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"CREATE TEMPORARY FUNCTION a(x FLOAT64, y FLOAT64) RETURNS FLOAT64 NOT DETERMINISTIC LANGUAGE js AS 'return x*y;'"
|
"CREATE TEMPORARY FUNCTION a(x FLOAT64, y FLOAT64) RETURNS FLOAT64 NOT DETERMINISTIC LANGUAGE js AS 'return x*y;'"
|
||||||
|
|
@ -1114,35 +1107,22 @@ WHERE
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_remove_precision_parameterized_types(self):
|
def test_remove_precision_parameterized_types(self):
|
||||||
self.validate_all(
|
self.validate_identity("CREATE TABLE test (a NUMERIC(10, 2))")
|
||||||
"SELECT CAST(1 AS NUMERIC(10, 2))",
|
self.validate_identity(
|
||||||
write={
|
|
||||||
"bigquery": "SELECT CAST(1 AS NUMERIC)",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"CREATE TABLE test (a NUMERIC(10, 2))",
|
|
||||||
write={
|
|
||||||
"bigquery": "CREATE TABLE test (a NUMERIC(10, 2))",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT CAST('1' AS STRING(10)) UNION ALL SELECT CAST('2' AS STRING(10))",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT CAST('1' AS STRING) UNION ALL SELECT CAST('2' AS STRING)",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT cola FROM (SELECT CAST('1' AS STRING(10)) AS cola UNION ALL SELECT CAST('2' AS STRING(10)) AS cola)",
|
|
||||||
write={
|
|
||||||
"bigquery": "SELECT cola FROM (SELECT CAST('1' AS STRING) AS cola UNION ALL SELECT CAST('2' AS STRING) AS cola)",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING(10)), CAST(14 AS STRING(10)))",
|
"INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING(10)), CAST(14 AS STRING(10)))",
|
||||||
write={
|
"INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING), CAST(14 AS STRING))",
|
||||||
"bigquery": "INSERT INTO test (cola, colb) VALUES (CAST(7 AS STRING), CAST(14 AS STRING))",
|
)
|
||||||
},
|
self.validate_identity(
|
||||||
|
"SELECT CAST(1 AS NUMERIC(10, 2))",
|
||||||
|
"SELECT CAST(1 AS NUMERIC)",
|
||||||
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"SELECT CAST('1' AS STRING(10)) UNION ALL SELECT CAST('2' AS STRING(10))",
|
||||||
|
"SELECT CAST('1' AS STRING) UNION ALL SELECT CAST('2' AS STRING)",
|
||||||
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"SELECT cola FROM (SELECT CAST('1' AS STRING(10)) AS cola UNION ALL SELECT CAST('2' AS STRING(10)) AS cola)",
|
||||||
|
"SELECT cola FROM (SELECT CAST('1' AS STRING) AS cola UNION ALL SELECT CAST('2' AS STRING) AS cola)",
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_models(self):
|
def test_models(self):
|
||||||
|
|
|
||||||
|
|
@ -369,6 +369,7 @@ class TestClickhouse(Validator):
|
||||||
"SELECT STARTS_WITH('a', 'b'), STARTSWITH('a', 'b')",
|
"SELECT STARTS_WITH('a', 'b'), STARTSWITH('a', 'b')",
|
||||||
write={"clickhouse": "SELECT startsWith('a', 'b'), startsWith('a', 'b')"},
|
write={"clickhouse": "SELECT startsWith('a', 'b'), startsWith('a', 'b')"},
|
||||||
)
|
)
|
||||||
|
self.validate_identity("SYSTEM STOP MERGES foo.bar", check_command_warning=True)
|
||||||
|
|
||||||
def test_cte(self):
|
def test_cte(self):
|
||||||
self.validate_identity("WITH 'x' AS foo SELECT foo")
|
self.validate_identity("WITH 'x' AS foo SELECT foo")
|
||||||
|
|
|
||||||
|
|
@ -2101,7 +2101,7 @@ SELECT
|
||||||
"databricks": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
"databricks": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
||||||
"presto": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
"presto": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
||||||
"snowflake": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
"snowflake": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
||||||
"sqlite": "SELECT SUM(CASE WHEN col % 2 = 0 THEN 1 ELSE 0 END) FROM foo",
|
"sqlite": "SELECT SUM(IIF(col % 2 = 0, 1, 0)) FROM foo",
|
||||||
"tsql": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
"tsql": "SELECT COUNT_IF(col % 2 = 0) FROM foo",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
@ -2116,7 +2116,7 @@ SELECT
|
||||||
"": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
"": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
||||||
"databricks": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
"databricks": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
||||||
"presto": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
"presto": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
||||||
"sqlite": "SELECT SUM(CASE WHEN col % 2 = 0 THEN 1 ELSE 0 END) FILTER(WHERE col < 1000) FROM foo",
|
"sqlite": "SELECT SUM(IIF(col % 2 = 0, 1, 0)) FILTER(WHERE col < 1000) FROM foo",
|
||||||
"tsql": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
"tsql": "SELECT COUNT_IF(col % 2 = 0) FILTER(WHERE col < 1000) FROM foo",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -152,7 +152,7 @@ class TestHive(Validator):
|
||||||
"duckdb": "CREATE TABLE x (w TEXT)", # Partition columns should exist in table
|
"duckdb": "CREATE TABLE x (w TEXT)", # Partition columns should exist in table
|
||||||
"presto": "CREATE TABLE x (w VARCHAR, y INTEGER, z INTEGER) WITH (PARTITIONED_BY=ARRAY['y', 'z'])",
|
"presto": "CREATE TABLE x (w VARCHAR, y INTEGER, z INTEGER) WITH (PARTITIONED_BY=ARRAY['y', 'z'])",
|
||||||
"hive": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
|
"hive": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
|
||||||
"spark": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
|
"spark": "CREATE TABLE x (w STRING, y INT, z INT) PARTITIONED BY (y, z)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,8 @@ class TestPostgres(Validator):
|
||||||
dialect = "postgres"
|
dialect = "postgres"
|
||||||
|
|
||||||
def test_postgres(self):
|
def test_postgres(self):
|
||||||
|
self.validate_identity("|/ x", "SQRT(x)")
|
||||||
|
self.validate_identity("||/ x", "CBRT(x)")
|
||||||
expr = parse_one(
|
expr = parse_one(
|
||||||
"SELECT * FROM r CROSS JOIN LATERAL UNNEST(ARRAY[1]) AS s(location)", read="postgres"
|
"SELECT * FROM r CROSS JOIN LATERAL UNNEST(ARRAY[1]) AS s(location)", read="postgres"
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -424,7 +424,7 @@ class TestPresto(Validator):
|
||||||
"duckdb": "CREATE TABLE x (w TEXT, y INT, z INT)",
|
"duckdb": "CREATE TABLE x (w TEXT, y INT, z INT)",
|
||||||
"presto": "CREATE TABLE x (w VARCHAR, y INTEGER, z INTEGER) WITH (PARTITIONED_BY=ARRAY['y', 'z'])",
|
"presto": "CREATE TABLE x (w VARCHAR, y INTEGER, z INTEGER) WITH (PARTITIONED_BY=ARRAY['y', 'z'])",
|
||||||
"hive": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
|
"hive": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
|
||||||
"spark": "CREATE TABLE x (w STRING) PARTITIONED BY (y INT, z INT)",
|
"spark": "CREATE TABLE x (w STRING, y INT, z INT) PARTITIONED BY (y, z)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
from sqlglot import transpile
|
from sqlglot import exp, parse_one, transpile
|
||||||
from tests.dialects.test_dialect import Validator
|
from tests.dialects.test_dialect import Validator
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -381,8 +381,6 @@ class TestRedshift(Validator):
|
||||||
"SELECT DATEADD(DAY, 1, DATE('2023-01-01'))",
|
"SELECT DATEADD(DAY, 1, DATE('2023-01-01'))",
|
||||||
)
|
)
|
||||||
|
|
||||||
self.validate_identity("SELECT * FROM x AS a, a.b AS c, c.d.e AS f, f.g.h.i.j.k AS l")
|
|
||||||
|
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"""SELECT
|
"""SELECT
|
||||||
c_name,
|
c_name,
|
||||||
|
|
@ -408,8 +406,9 @@ ORDER BY
|
||||||
union_query = f"SELECT * FROM ({' UNION ALL '.join('SELECT ' + v for v in values)})"
|
union_query = f"SELECT * FROM ({' UNION ALL '.join('SELECT ' + v for v in values)})"
|
||||||
self.assertEqual(transpile(values_query, write="redshift")[0], union_query)
|
self.assertEqual(transpile(values_query, write="redshift")[0], union_query)
|
||||||
|
|
||||||
self.validate_identity(
|
values_sql = transpile("SELECT * FROM (VALUES (1), (2))", write="redshift", pretty=True)[0]
|
||||||
"SELECT * FROM (VALUES (1), (2))",
|
self.assertEqual(
|
||||||
|
values_sql,
|
||||||
"""SELECT
|
"""SELECT
|
||||||
*
|
*
|
||||||
FROM (
|
FROM (
|
||||||
|
|
@ -419,69 +418,51 @@ FROM (
|
||||||
SELECT
|
SELECT
|
||||||
2
|
2
|
||||||
)""",
|
)""",
|
||||||
pretty=True,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self.validate_identity("INSERT INTO t (a) VALUES (1), (2), (3)")
|
||||||
|
self.validate_identity("INSERT INTO t (a, b) VALUES (1, 2), (3, 4)")
|
||||||
|
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT * FROM (VALUES (1, 2)) AS t",
|
"SELECT * FROM (SELECT 1, 2) AS t",
|
||||||
|
read={
|
||||||
|
"": "SELECT * FROM (VALUES (1, 2)) AS t",
|
||||||
|
},
|
||||||
write={
|
write={
|
||||||
"redshift": "SELECT * FROM (SELECT 1, 2) AS t",
|
|
||||||
"mysql": "SELECT * FROM (SELECT 1, 2) AS t",
|
"mysql": "SELECT * FROM (SELECT 1, 2) AS t",
|
||||||
"presto": "SELECT * FROM (VALUES (1, 2)) AS t",
|
"presto": "SELECT * FROM (SELECT 1, 2) AS t",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
|
"SELECT * FROM (SELECT 1 AS id) AS t1 CROSS JOIN (SELECT 1 AS id) AS t2",
|
||||||
write={
|
read={
|
||||||
"redshift": "SELECT * FROM (SELECT 1 AS id) AS t1 CROSS JOIN (SELECT 1 AS id) AS t2",
|
"": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT a, b FROM (VALUES (1, 2)) AS t (a, b)",
|
"SELECT a, b FROM (SELECT 1 AS a, 2 AS b) AS t",
|
||||||
write={
|
read={
|
||||||
"redshift": "SELECT a, b FROM (SELECT 1 AS a, 2 AS b) AS t",
|
"": "SELECT a, b FROM (VALUES (1, 2)) AS t (a, b)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
'SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS "t" (a, b)',
|
'SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS "t"',
|
||||||
write={
|
read={
|
||||||
"redshift": 'SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS "t"',
|
"": 'SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS "t" (a, b)',
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT a, b FROM (VALUES (1, 2), (3, 4), (5, 6), (7, 8)) AS t (a, b)",
|
"SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4 UNION ALL SELECT 5, 6 UNION ALL SELECT 7, 8) AS t",
|
||||||
write={
|
read={
|
||||||
"redshift": "SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4 UNION ALL SELECT 5, 6 UNION ALL SELECT 7, 8) AS t",
|
"": "SELECT a, b FROM (VALUES (1, 2), (3, 4), (5, 6), (7, 8)) AS t (a, b)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"INSERT INTO t(a) VALUES (1), (2), (3)",
|
"INSERT INTO t (a, b) SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS t",
|
||||||
write={
|
read={
|
||||||
"redshift": "INSERT INTO t (a) VALUES (1), (2), (3)",
|
"": "INSERT INTO t(a, b) SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS t (a, b)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
|
||||||
"INSERT INTO t(a, b) SELECT a, b FROM (VALUES (1, 2), (3, 4)) AS t (a, b)",
|
|
||||||
write={
|
|
||||||
"redshift": "INSERT INTO t (a, b) SELECT a, b FROM (SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4) AS t",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"INSERT INTO t(a, b) VALUES (1, 2), (3, 4)",
|
|
||||||
write={
|
|
||||||
"redshift": "INSERT INTO t (a, b) VALUES (1, 2), (3, 4)",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
self.validate_identity(
|
|
||||||
'SELECT * FROM (VALUES (1)) AS "t"(a)',
|
|
||||||
'''SELECT
|
|
||||||
*
|
|
||||||
FROM (
|
|
||||||
SELECT
|
|
||||||
1 AS a
|
|
||||||
) AS "t"''',
|
|
||||||
pretty=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_create_table_like(self):
|
def test_create_table_like(self):
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
|
|
@ -532,3 +513,26 @@ FROM (
|
||||||
"redshift": "CREATE OR REPLACE VIEW v1 AS SELECT cola, colb FROM t1 WITH NO SCHEMA BINDING",
|
"redshift": "CREATE OR REPLACE VIEW v1 AS SELECT cola, colb FROM t1 WITH NO SCHEMA BINDING",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_column_unnesting(self):
|
||||||
|
ast = parse_one("SELECT * FROM t.t JOIN t.c1 ON c1.c2 = t.c3", read="redshift")
|
||||||
|
ast.args["from"].this.assert_is(exp.Table)
|
||||||
|
ast.args["joins"][0].this.assert_is(exp.Table)
|
||||||
|
self.assertEqual(ast.sql("redshift"), "SELECT * FROM t.t JOIN t.c1 ON c1.c2 = t.c3")
|
||||||
|
|
||||||
|
ast = parse_one("SELECT * FROM t AS t CROSS JOIN t.c1", read="redshift")
|
||||||
|
ast.args["from"].this.assert_is(exp.Table)
|
||||||
|
ast.args["joins"][0].this.assert_is(exp.Column)
|
||||||
|
self.assertEqual(ast.sql("redshift"), "SELECT * FROM t AS t CROSS JOIN t.c1")
|
||||||
|
|
||||||
|
ast = parse_one(
|
||||||
|
"SELECT * FROM x AS a, a.b AS c, c.d.e AS f, f.g.h.i.j.k AS l", read="redshift"
|
||||||
|
)
|
||||||
|
joins = ast.args["joins"]
|
||||||
|
ast.args["from"].this.assert_is(exp.Table)
|
||||||
|
joins[0].this.this.assert_is(exp.Column)
|
||||||
|
joins[1].this.this.assert_is(exp.Column)
|
||||||
|
joins[2].this.this.assert_is(exp.Dot)
|
||||||
|
self.assertEqual(
|
||||||
|
ast.sql("redshift"), "SELECT * FROM x AS a, a.b AS c, c.d.e AS f, f.g.h.i.j.k AS l"
|
||||||
|
)
|
||||||
|
|
|
||||||
|
|
@ -84,6 +84,10 @@ WHERE
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"SELECT a FROM test PIVOT(SUM(x) FOR y IN ('z', 'q')) AS x TABLESAMPLE (0.1)"
|
"SELECT a FROM test PIVOT(SUM(x) FOR y IN ('z', 'q')) AS x TABLESAMPLE (0.1)"
|
||||||
)
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"value:values::string",
|
||||||
|
"CAST(GET_PATH(value, 'values') AS TEXT)",
|
||||||
|
)
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"""SELECT GET_PATH(PARSE_JSON('{"y": [{"z": 1}]}'), 'y[0]:z')""",
|
"""SELECT GET_PATH(PARSE_JSON('{"y": [{"z": 1}]}'), 'y[0]:z')""",
|
||||||
"""SELECT GET_PATH(PARSE_JSON('{"y": [{"z": 1}]}'), 'y[0].z')""",
|
"""SELECT GET_PATH(PARSE_JSON('{"y": [{"z": 1}]}'), 'y[0].z')""",
|
||||||
|
|
@ -462,7 +466,7 @@ WHERE
|
||||||
"DIV0(foo, bar)",
|
"DIV0(foo, bar)",
|
||||||
write={
|
write={
|
||||||
"snowflake": "IFF(bar = 0, 0, foo / bar)",
|
"snowflake": "IFF(bar = 0, 0, foo / bar)",
|
||||||
"sqlite": "CASE WHEN bar = 0 THEN 0 ELSE CAST(foo AS REAL) / bar END",
|
"sqlite": "IIF(bar = 0, 0, CAST(foo AS REAL) / bar)",
|
||||||
"presto": "IF(bar = 0, 0, CAST(foo AS DOUBLE) / bar)",
|
"presto": "IF(bar = 0, 0, CAST(foo AS DOUBLE) / bar)",
|
||||||
"spark": "IF(bar = 0, 0, foo / bar)",
|
"spark": "IF(bar = 0, 0, foo / bar)",
|
||||||
"hive": "IF(bar = 0, 0, foo / bar)",
|
"hive": "IF(bar = 0, 0, foo / bar)",
|
||||||
|
|
@ -473,7 +477,7 @@ WHERE
|
||||||
"ZEROIFNULL(foo)",
|
"ZEROIFNULL(foo)",
|
||||||
write={
|
write={
|
||||||
"snowflake": "IFF(foo IS NULL, 0, foo)",
|
"snowflake": "IFF(foo IS NULL, 0, foo)",
|
||||||
"sqlite": "CASE WHEN foo IS NULL THEN 0 ELSE foo END",
|
"sqlite": "IIF(foo IS NULL, 0, foo)",
|
||||||
"presto": "IF(foo IS NULL, 0, foo)",
|
"presto": "IF(foo IS NULL, 0, foo)",
|
||||||
"spark": "IF(foo IS NULL, 0, foo)",
|
"spark": "IF(foo IS NULL, 0, foo)",
|
||||||
"hive": "IF(foo IS NULL, 0, foo)",
|
"hive": "IF(foo IS NULL, 0, foo)",
|
||||||
|
|
@ -484,7 +488,7 @@ WHERE
|
||||||
"NULLIFZERO(foo)",
|
"NULLIFZERO(foo)",
|
||||||
write={
|
write={
|
||||||
"snowflake": "IFF(foo = 0, NULL, foo)",
|
"snowflake": "IFF(foo = 0, NULL, foo)",
|
||||||
"sqlite": "CASE WHEN foo = 0 THEN NULL ELSE foo END",
|
"sqlite": "IIF(foo = 0, NULL, foo)",
|
||||||
"presto": "IF(foo = 0, NULL, foo)",
|
"presto": "IF(foo = 0, NULL, foo)",
|
||||||
"spark": "IF(foo = 0, NULL, foo)",
|
"spark": "IF(foo = 0, NULL, foo)",
|
||||||
"hive": "IF(foo = 0, NULL, foo)",
|
"hive": "IF(foo = 0, NULL, foo)",
|
||||||
|
|
@ -1513,6 +1517,10 @@ MATCH_RECOGNIZE (
|
||||||
self.validate_identity("SHOW COLUMNS IN VIEW")
|
self.validate_identity("SHOW COLUMNS IN VIEW")
|
||||||
self.validate_identity("SHOW COLUMNS LIKE '_foo%' IN VIEW dt_test")
|
self.validate_identity("SHOW COLUMNS LIKE '_foo%' IN VIEW dt_test")
|
||||||
|
|
||||||
|
self.validate_identity("SHOW USERS")
|
||||||
|
self.validate_identity("SHOW TERSE USERS")
|
||||||
|
self.validate_identity("SHOW USERS LIKE '_foo%' STARTS WITH 'bar' LIMIT 5 FROM 'baz'")
|
||||||
|
|
||||||
ast = parse_one("SHOW COLUMNS LIKE '_testing%' IN dt_test", read="snowflake")
|
ast = parse_one("SHOW COLUMNS LIKE '_testing%' IN dt_test", read="snowflake")
|
||||||
table = ast.find(exp.Table)
|
table = ast.find(exp.Table)
|
||||||
literal = ast.find(exp.Literal)
|
literal = ast.find(exp.Literal)
|
||||||
|
|
@ -1536,6 +1544,10 @@ MATCH_RECOGNIZE (
|
||||||
table = ast.find(exp.Table)
|
table = ast.find(exp.Table)
|
||||||
self.assertEqual(table.sql(dialect="snowflake"), "db1.schema1")
|
self.assertEqual(table.sql(dialect="snowflake"), "db1.schema1")
|
||||||
|
|
||||||
|
users_exp = self.validate_identity("SHOW USERS")
|
||||||
|
self.assertTrue(isinstance(users_exp, exp.Show))
|
||||||
|
self.assertEqual(users_exp.this, "USERS")
|
||||||
|
|
||||||
def test_swap(self):
|
def test_swap(self):
|
||||||
ast = parse_one("ALTER TABLE a SWAP WITH b", read="snowflake")
|
ast = parse_one("ALTER TABLE a SWAP WITH b", read="snowflake")
|
||||||
assert isinstance(ast, exp.AlterTable)
|
assert isinstance(ast, exp.AlterTable)
|
||||||
|
|
|
||||||
|
|
@ -93,11 +93,12 @@ TBLPROPERTIES (
|
||||||
'x'='1'
|
'x'='1'
|
||||||
)""",
|
)""",
|
||||||
"spark": """CREATE TABLE blah (
|
"spark": """CREATE TABLE blah (
|
||||||
col_a INT
|
col_a INT,
|
||||||
|
date STRING
|
||||||
)
|
)
|
||||||
COMMENT 'Test comment: blah'
|
COMMENT 'Test comment: blah'
|
||||||
PARTITIONED BY (
|
PARTITIONED BY (
|
||||||
date STRING
|
date
|
||||||
)
|
)
|
||||||
USING ICEBERG
|
USING ICEBERG
|
||||||
TBLPROPERTIES (
|
TBLPROPERTIES (
|
||||||
|
|
@ -125,13 +126,6 @@ TBLPROPERTIES (
|
||||||
"spark": "ALTER TABLE StudentInfo DROP COLUMNS (LastName, DOB)",
|
"spark": "ALTER TABLE StudentInfo DROP COLUMNS (LastName, DOB)",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
self.validate_all(
|
|
||||||
"CREATE TABLE x USING ICEBERG PARTITIONED BY (MONTHS(y)) LOCATION 's3://z'",
|
|
||||||
identify=True,
|
|
||||||
write={
|
|
||||||
"spark": "CREATE TABLE `x` USING ICEBERG PARTITIONED BY (MONTHS(`y`)) LOCATION 's3://z'",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_to_date(self):
|
def test_to_date(self):
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
|
|
@ -256,6 +250,14 @@ TBLPROPERTIES (
|
||||||
self.validate_identity("TRIM(LEADING 'SL' FROM 'SSparkSQLS')")
|
self.validate_identity("TRIM(LEADING 'SL' FROM 'SSparkSQLS')")
|
||||||
self.validate_identity("TRIM(TRAILING 'SL' FROM 'SSparkSQLS')")
|
self.validate_identity("TRIM(TRAILING 'SL' FROM 'SSparkSQLS')")
|
||||||
self.validate_identity("SPLIT(str, pattern, lim)")
|
self.validate_identity("SPLIT(str, pattern, lim)")
|
||||||
|
self.validate_identity(
|
||||||
|
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL 23 HOUR + 59 MINUTE + 59 SECONDS",
|
||||||
|
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL '23' HOUR + INTERVAL '59' MINUTE + INTERVAL '59' SECONDS",
|
||||||
|
)
|
||||||
|
self.validate_identity(
|
||||||
|
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL '23' HOUR + '59' MINUTE + '59' SECONDS",
|
||||||
|
"SELECT CAST('2023-01-01' AS TIMESTAMP) + INTERVAL '23' HOUR + INTERVAL '59' MINUTE + INTERVAL '59' SECONDS",
|
||||||
|
)
|
||||||
self.validate_identity(
|
self.validate_identity(
|
||||||
"SELECT INTERVAL '5' HOURS '30' MINUTES '5' SECONDS '6' MILLISECONDS '7' MICROSECONDS",
|
"SELECT INTERVAL '5' HOURS '30' MINUTES '5' SECONDS '6' MILLISECONDS '7' MICROSECONDS",
|
||||||
"SELECT INTERVAL '5' HOURS + INTERVAL '30' MINUTES + INTERVAL '5' SECONDS + INTERVAL '6' MILLISECONDS + INTERVAL '7' MICROSECONDS",
|
"SELECT INTERVAL '5' HOURS + INTERVAL '30' MINUTES + INTERVAL '5' SECONDS + INTERVAL '6' MILLISECONDS + INTERVAL '7' MICROSECONDS",
|
||||||
|
|
@ -616,12 +618,6 @@ TBLPROPERTIES (
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_iif(self):
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT IIF(cond, 'True', 'False')",
|
|
||||||
write={"spark": "SELECT IF(cond, 'True', 'False')"},
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_bool_or(self):
|
def test_bool_or(self):
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT a, LOGICAL_OR(b) FROM table GROUP BY a",
|
"SELECT a, LOGICAL_OR(b) FROM table GROUP BY a",
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,21 @@ class TestTableau(Validator):
|
||||||
dialect = "tableau"
|
dialect = "tableau"
|
||||||
|
|
||||||
def test_tableau(self):
|
def test_tableau(self):
|
||||||
|
self.validate_all(
|
||||||
|
"[x]",
|
||||||
|
write={
|
||||||
|
"hive": "`x`",
|
||||||
|
"tableau": "[x]",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.validate_all(
|
||||||
|
'"x"',
|
||||||
|
write={
|
||||||
|
"hive": "'x'",
|
||||||
|
"tableau": "'x'",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"IF x = 'a' THEN y ELSE NULL END",
|
"IF x = 'a' THEN y ELSE NULL END",
|
||||||
read={
|
read={
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,14 @@ class TestTSQL(Validator):
|
||||||
self.validate_identity("1 AND true", "1 <> 0 AND (1 = 1)")
|
self.validate_identity("1 AND true", "1 <> 0 AND (1 = 1)")
|
||||||
self.validate_identity("CAST(x AS int) OR y", "CAST(x AS INTEGER) <> 0 OR y <> 0")
|
self.validate_identity("CAST(x AS int) OR y", "CAST(x AS INTEGER) <> 0 OR y <> 0")
|
||||||
|
|
||||||
|
self.validate_all(
|
||||||
|
"SELECT IIF(cond <> 0, 'True', 'False')",
|
||||||
|
read={
|
||||||
|
"spark": "SELECT IF(cond, 'True', 'False')",
|
||||||
|
"sqlite": "SELECT IIF(cond, 'True', 'False')",
|
||||||
|
"tsql": "SELECT IIF(cond <> 0, 'True', 'False')",
|
||||||
|
},
|
||||||
|
)
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT TRIM(BOTH 'a' FROM a)",
|
"SELECT TRIM(BOTH 'a' FROM a)",
|
||||||
read={
|
read={
|
||||||
|
|
@ -1302,20 +1310,6 @@ WHERE
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_iif(self):
|
|
||||||
self.validate_identity(
|
|
||||||
"SELECT IF(cond, 'True', 'False')", "SELECT IIF(cond <> 0, 'True', 'False')"
|
|
||||||
)
|
|
||||||
self.validate_identity(
|
|
||||||
"SELECT IIF(cond, 'True', 'False')", "SELECT IIF(cond <> 0, 'True', 'False')"
|
|
||||||
)
|
|
||||||
self.validate_all(
|
|
||||||
"SELECT IIF(cond, 'True', 'False');",
|
|
||||||
write={
|
|
||||||
"spark": "SELECT IF(cond, 'True', 'False')",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
def test_lateral_subquery(self):
|
def test_lateral_subquery(self):
|
||||||
self.validate_all(
|
self.validate_all(
|
||||||
"SELECT x.a, x.b, t.v, t.y FROM x CROSS APPLY (SELECT v, y FROM t) t(v, y)",
|
"SELECT x.a, x.b, t.v, t.y FROM x CROSS APPLY (SELECT v, y FROM t) t(v, y)",
|
||||||
|
|
|
||||||
3
tests/fixtures/identity.sql
vendored
3
tests/fixtures/identity.sql
vendored
|
|
@ -847,3 +847,6 @@ WITH use(use) AS (SELECT 1) SELECT use FROM use
|
||||||
SELECT recursive FROM t
|
SELECT recursive FROM t
|
||||||
SELECT (ROW_NUMBER() OVER (PARTITION BY user ORDER BY date ASC) - ROW_NUMBER() OVER (PARTITION BY user, segment ORDER BY date ASC)) AS group_id FROM example_table
|
SELECT (ROW_NUMBER() OVER (PARTITION BY user ORDER BY date ASC) - ROW_NUMBER() OVER (PARTITION BY user, segment ORDER BY date ASC)) AS group_id FROM example_table
|
||||||
CAST(foo AS BPCHAR)
|
CAST(foo AS BPCHAR)
|
||||||
|
values
|
||||||
|
SELECT values
|
||||||
|
SELECT values AS values FROM t WHERE values + 1 > 3
|
||||||
|
|
|
||||||
12
tests/fixtures/optimizer/canonicalize.sql
vendored
12
tests/fixtures/optimizer/canonicalize.sql
vendored
|
|
@ -96,3 +96,15 @@ DATE_TRUNC('DAY', CAST('2023-01-01' AS DATE));
|
||||||
|
|
||||||
DATEDIFF('2023-01-01', '2023-01-02', DAY);
|
DATEDIFF('2023-01-01', '2023-01-02', DAY);
|
||||||
DATEDIFF(CAST('2023-01-01' AS DATETIME), CAST('2023-01-02' AS DATETIME), DAY);
|
DATEDIFF(CAST('2023-01-01' AS DATETIME), CAST('2023-01-02' AS DATETIME), DAY);
|
||||||
|
|
||||||
|
--------------------------------------
|
||||||
|
-- Remove redundant casts
|
||||||
|
--------------------------------------
|
||||||
|
CAST(CAST('2023-01-01' AS DATE) AS DATE);
|
||||||
|
CAST('2023-01-01' AS DATE);
|
||||||
|
|
||||||
|
CAST(DATE_TRUNC('YEAR', CAST('2023-01-01' AS DATE)) AS DATE);
|
||||||
|
DATE_TRUNC('YEAR', CAST('2023-01-01' AS DATE));
|
||||||
|
|
||||||
|
DATE(DATE_TRUNC('YEAR', CAST("x" AS DATE)));
|
||||||
|
DATE_TRUNC('YEAR', CAST("x" AS DATE));
|
||||||
|
|
|
||||||
|
|
@ -218,6 +218,7 @@ with t1 as (
|
||||||
ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num
|
ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num
|
||||||
FROM
|
FROM
|
||||||
x
|
x
|
||||||
|
ORDER BY x.a, x.b, row_num
|
||||||
)
|
)
|
||||||
SELECT
|
SELECT
|
||||||
t1.a,
|
t1.a,
|
||||||
|
|
@ -226,7 +227,7 @@ FROM
|
||||||
t1
|
t1
|
||||||
WHERE
|
WHERE
|
||||||
row_num = 1;
|
row_num = 1;
|
||||||
WITH t1 AS (SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x) SELECT t1.a AS a, t1.b AS b FROM t1 AS t1 WHERE t1.row_num = 1;
|
WITH t1 AS (SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x ORDER BY x.a, x.b, row_num) SELECT t1.a AS a, t1.b AS b FROM t1 AS t1 WHERE t1.row_num = 1;
|
||||||
|
|
||||||
# title: Test preventing merge of window expressions join clause
|
# title: Test preventing merge of window expressions join clause
|
||||||
with t1 as (
|
with t1 as (
|
||||||
|
|
@ -301,6 +302,7 @@ with t1 as (
|
||||||
ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num
|
ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num
|
||||||
FROM
|
FROM
|
||||||
x
|
x
|
||||||
|
ORDER BY x.a, x.b, row_num
|
||||||
)
|
)
|
||||||
SELECT
|
SELECT
|
||||||
t1.a,
|
t1.a,
|
||||||
|
|
@ -308,7 +310,7 @@ SELECT
|
||||||
t1.row_num
|
t1.row_num
|
||||||
FROM
|
FROM
|
||||||
t1;
|
t1;
|
||||||
SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x;
|
SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x ORDER BY x.a, x.b, row_num;
|
||||||
|
|
||||||
# title: Don't merge window functions, inner table is aliased in outer query
|
# title: Don't merge window functions, inner table is aliased in outer query
|
||||||
with t1 as (
|
with t1 as (
|
||||||
|
|
|
||||||
32
tests/fixtures/optimizer/qualify_columns.sql
vendored
32
tests/fixtures/optimizer/qualify_columns.sql
vendored
|
|
@ -208,14 +208,14 @@ SELECT x.a AS a, y.c AS c FROM x AS x, y AS y;
|
||||||
--------------------------------------
|
--------------------------------------
|
||||||
-- Unions
|
-- Unions
|
||||||
--------------------------------------
|
--------------------------------------
|
||||||
SELECT a FROM x UNION SELECT a FROM x;
|
SELECT a FROM x UNION SELECT a FROM x ORDER BY a;
|
||||||
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x;
|
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x ORDER BY a;
|
||||||
|
|
||||||
SELECT a FROM x UNION SELECT a FROM x UNION SELECT a FROM x;
|
SELECT a FROM x UNION SELECT a FROM x UNION SELECT a FROM x ORDER BY a;
|
||||||
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x;
|
SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x ORDER BY a;
|
||||||
|
|
||||||
SELECT a FROM (SELECT a FROM x UNION SELECT a FROM x);
|
SELECT a FROM (SELECT a FROM x UNION SELECT a FROM x) ORDER BY a;
|
||||||
SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x) AS _q_0;
|
SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM x AS x) AS _q_0 ORDER BY a;
|
||||||
|
|
||||||
--------------------------------------
|
--------------------------------------
|
||||||
-- Subqueries
|
-- Subqueries
|
||||||
|
|
@ -318,8 +318,8 @@ WITH z AS (SELECT x.a AS a FROM x AS x) SELECT z.a AS a FROM z AS z;
|
||||||
WITH z AS (SELECT a FROM x), q AS (SELECT * FROM z) SELECT * FROM q;
|
WITH z AS (SELECT a FROM x), q AS (SELECT * FROM z) SELECT * FROM q;
|
||||||
WITH z AS (SELECT x.a AS a FROM x AS x), q AS (SELECT z.a AS a FROM z AS z) SELECT q.a AS a FROM q AS q;
|
WITH z AS (SELECT x.a AS a FROM x AS x), q AS (SELECT z.a AS a FROM z AS z) SELECT q.a AS a FROM q AS q;
|
||||||
|
|
||||||
WITH z AS (SELECT * FROM x) SELECT * FROM z UNION SELECT * FROM z;
|
WITH z AS (SELECT * FROM x) SELECT * FROM z UNION SELECT * FROM z ORDER BY a, b;
|
||||||
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x) SELECT z.a AS a, z.b AS b FROM z AS z UNION SELECT z.a AS a, z.b AS b FROM z AS z;
|
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x) SELECT z.a AS a, z.b AS b FROM z AS z UNION SELECT z.a AS a, z.b AS b FROM z AS z ORDER BY a, b;
|
||||||
|
|
||||||
WITH z AS (SELECT * FROM x), q AS (SELECT b FROM z) SELECT b FROM q;
|
WITH z AS (SELECT * FROM x), q AS (SELECT b FROM z) SELECT b FROM q;
|
||||||
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x), q AS (SELECT z.b AS b FROM z AS z) SELECT q.b AS b FROM q AS q;
|
WITH z AS (SELECT x.a AS a, x.b AS b FROM x AS x), q AS (SELECT z.b AS b FROM z AS z) SELECT q.b AS b FROM q AS q;
|
||||||
|
|
@ -359,8 +359,8 @@ SELECT x.b AS b FROM x AS x;
|
||||||
SELECT * EXCEPT (a, b) FROM x;
|
SELECT * EXCEPT (a, b) FROM x;
|
||||||
SELECT * EXCEPT (x.a, x.b) FROM x AS x;
|
SELECT * EXCEPT (x.a, x.b) FROM x AS x;
|
||||||
|
|
||||||
SELECT COALESCE(t1.a, '') AS a, t2.* EXCEPT (a) FROM x AS t1, x AS t2;
|
SELECT COALESCE(CAST(t1.a AS VARCHAR), '') AS a, t2.* EXCEPT (a) FROM x AS t1, x AS t2;
|
||||||
SELECT COALESCE(t1.a, '') AS a, t2.b AS b FROM x AS t1, x AS t2;
|
SELECT COALESCE(CAST(t1.a AS VARCHAR), '') AS a, t2.b AS b FROM x AS t1, x AS t2;
|
||||||
|
|
||||||
--------------------------------------
|
--------------------------------------
|
||||||
-- Using
|
-- Using
|
||||||
|
|
@ -468,8 +468,8 @@ select * from unnest ([1, 2]) as x with offset as y;
|
||||||
SELECT x AS x, y AS y FROM UNNEST([1, 2]) AS x WITH OFFSET AS y;
|
SELECT x AS x, y AS y FROM UNNEST([1, 2]) AS x WITH OFFSET AS y;
|
||||||
|
|
||||||
# dialect: presto
|
# dialect: presto
|
||||||
SELECT x.a, i.b FROM x CROSS JOIN UNNEST(SPLIT(b, ',')) AS i(b);
|
SELECT x.a, i.b FROM x CROSS JOIN UNNEST(SPLIT(CAST(b AS VARCHAR), ',')) AS i(b);
|
||||||
SELECT x.a AS a, i.b AS b FROM x AS x CROSS JOIN UNNEST(SPLIT(x.b, ',')) AS i(b);
|
SELECT x.a AS a, i.b AS b FROM x AS x CROSS JOIN UNNEST(SPLIT(CAST(x.b AS VARCHAR), ',')) AS i(b);
|
||||||
|
|
||||||
# execute: false
|
# execute: false
|
||||||
SELECT c FROM (SELECT 1 a) AS x LATERAL VIEW EXPLODE(a) AS c;
|
SELECT c FROM (SELECT 1 a) AS x LATERAL VIEW EXPLODE(a) AS c;
|
||||||
|
|
@ -487,16 +487,16 @@ SELECT t.c1 AS c1, t.c3 AS c3 FROM FOO(bar) AS t(c1, c2, c3);
|
||||||
-- Window functions
|
-- Window functions
|
||||||
--------------------------------------
|
--------------------------------------
|
||||||
# title: ORDER BY in window function
|
# title: ORDER BY in window function
|
||||||
SELECT a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY b ORDER BY a) AS row_num FROM x;
|
SELECT a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY b ORDER BY a) AS row_num FROM x ORDER BY a, row_num;
|
||||||
SELECT x.a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a) AS row_num FROM x AS x;
|
SELECT x.a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a) AS row_num FROM x AS x ORDER BY a, row_num;
|
||||||
|
|
||||||
# dialect: bigquery
|
# dialect: bigquery
|
||||||
SELECT ROW_NUMBER() OVER (PARTITION BY a ORDER BY b) AS row_num FROM x QUALIFY row_num = 1;
|
SELECT ROW_NUMBER() OVER (PARTITION BY a ORDER BY b) AS row_num FROM x QUALIFY row_num = 1;
|
||||||
SELECT ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.b) AS row_num FROM x AS x QUALIFY ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.b) = 1;
|
SELECT ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.b) AS row_num FROM x AS x QUALIFY ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.b) = 1;
|
||||||
|
|
||||||
# dialect: bigquery
|
# dialect: bigquery
|
||||||
SELECT x.b, x.a FROM x LEFT JOIN y ON x.b = y.b QUALIFY ROW_NUMBER() OVER(PARTITION BY x.b ORDER BY x.a DESC) = 1;
|
SELECT x.b, x.a FROM x LEFT JOIN y ON x.b = y.b QUALIFY ROW_NUMBER() OVER(PARTITION BY x.b ORDER BY x.a DESC) = 1 ORDER BY x.b, x.a;
|
||||||
SELECT x.b AS b, x.a AS a FROM x AS x LEFT JOIN y AS y ON x.b = y.b QUALIFY ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a DESC) = 1;
|
SELECT x.b AS b, x.a AS a FROM x AS x LEFT JOIN y AS y ON x.b = y.b QUALIFY ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a DESC) = 1 ORDER BY x.b, x.a;
|
||||||
|
|
||||||
SELECT * FROM x QUALIFY COUNT(a) OVER (PARTITION BY b) > 1;
|
SELECT * FROM x QUALIFY COUNT(a) OVER (PARTITION BY b) > 1;
|
||||||
SELECT x.a AS a, x.b AS b FROM x AS x QUALIFY COUNT(x.a) OVER (PARTITION BY x.b) > 1;
|
SELECT x.a AS a, x.b AS b FROM x AS x QUALIFY COUNT(x.a) OVER (PARTITION BY x.b) > 1;
|
||||||
|
|
|
||||||
38
tests/fixtures/optimizer/tpc-h/tpc-h.sql
vendored
38
tests/fixtures/optimizer/tpc-h/tpc-h.sql
vendored
|
|
@ -15,7 +15,7 @@ select
|
||||||
from
|
from
|
||||||
lineitem
|
lineitem
|
||||||
where
|
where
|
||||||
l_shipdate <= date '1998-12-01' - interval '90' day
|
CAST(l_shipdate AS DATE) <= date '1998-12-01' - interval '90' day
|
||||||
group by
|
group by
|
||||||
l_returnflag,
|
l_returnflag,
|
||||||
l_linestatus
|
l_linestatus
|
||||||
|
|
@ -218,8 +218,8 @@ select
|
||||||
from
|
from
|
||||||
orders
|
orders
|
||||||
where
|
where
|
||||||
o_orderdate >= date '1993-07-01'
|
CAST(o_orderdate AS DATE) >= date '1993-07-01'
|
||||||
and o_orderdate < date '1993-07-01' + interval '3' month
|
and CAST(o_orderdate AS DATE) < date '1993-07-01' + interval '3' month
|
||||||
and exists (
|
and exists (
|
||||||
select
|
select
|
||||||
*
|
*
|
||||||
|
|
@ -278,8 +278,8 @@ where
|
||||||
and s_nationkey = n_nationkey
|
and s_nationkey = n_nationkey
|
||||||
and n_regionkey = r_regionkey
|
and n_regionkey = r_regionkey
|
||||||
and r_name = 'ASIA'
|
and r_name = 'ASIA'
|
||||||
and o_orderdate >= date '1994-01-01'
|
and CAST(o_orderdate AS DATE) >= date '1994-01-01'
|
||||||
and o_orderdate < date '1994-01-01' + interval '1' year
|
and CAST(o_orderdate AS DATE) < date '1994-01-01' + interval '1' year
|
||||||
group by
|
group by
|
||||||
n_name
|
n_name
|
||||||
order by
|
order by
|
||||||
|
|
@ -316,8 +316,8 @@ select
|
||||||
from
|
from
|
||||||
lineitem
|
lineitem
|
||||||
where
|
where
|
||||||
l_shipdate >= date '1994-01-01'
|
CAST(l_shipdate AS DATE) >= date '1994-01-01'
|
||||||
and l_shipdate < date '1994-01-01' + interval '1' year
|
and CAST(l_shipdate AS DATE) < date '1994-01-01' + interval '1' year
|
||||||
and l_discount between 0.06 - 0.01 and 0.06 + 0.01
|
and l_discount between 0.06 - 0.01 and 0.06 + 0.01
|
||||||
and l_quantity < 24;
|
and l_quantity < 24;
|
||||||
SELECT
|
SELECT
|
||||||
|
|
@ -362,7 +362,7 @@ from
|
||||||
(n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
|
(n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
|
||||||
or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
|
or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
|
||||||
)
|
)
|
||||||
and l_shipdate between date '1995-01-01' and date '1996-12-31'
|
and CAST(l_shipdate AS DATE) between date '1995-01-01' and date '1996-12-31'
|
||||||
) as shipping
|
) as shipping
|
||||||
group by
|
group by
|
||||||
supp_nation,
|
supp_nation,
|
||||||
|
|
@ -446,7 +446,7 @@ from
|
||||||
and n1.n_regionkey = r_regionkey
|
and n1.n_regionkey = r_regionkey
|
||||||
and r_name = 'AMERICA'
|
and r_name = 'AMERICA'
|
||||||
and s_nationkey = n2.n_nationkey
|
and s_nationkey = n2.n_nationkey
|
||||||
and o_orderdate between date '1995-01-01' and date '1996-12-31'
|
and CAST(o_orderdate AS DATE) between date '1995-01-01' and date '1996-12-31'
|
||||||
and p_type = 'ECONOMY ANODIZED STEEL'
|
and p_type = 'ECONOMY ANODIZED STEEL'
|
||||||
) as all_nations
|
) as all_nations
|
||||||
group by
|
group by
|
||||||
|
|
@ -574,8 +574,8 @@ from
|
||||||
where
|
where
|
||||||
c_custkey = o_custkey
|
c_custkey = o_custkey
|
||||||
and l_orderkey = o_orderkey
|
and l_orderkey = o_orderkey
|
||||||
and o_orderdate >= date '1993-10-01'
|
and CAST(o_orderdate AS DATE) >= date '1993-10-01'
|
||||||
and o_orderdate < date '1993-10-01' + interval '3' month
|
and CAST(o_orderdate AS DATE) < date '1993-10-01' + interval '3' month
|
||||||
and l_returnflag = 'R'
|
and l_returnflag = 'R'
|
||||||
and c_nationkey = n_nationkey
|
and c_nationkey = n_nationkey
|
||||||
group by
|
group by
|
||||||
|
|
@ -714,8 +714,8 @@ where
|
||||||
and l_shipmode in ('MAIL', 'SHIP')
|
and l_shipmode in ('MAIL', 'SHIP')
|
||||||
and l_commitdate < l_receiptdate
|
and l_commitdate < l_receiptdate
|
||||||
and l_shipdate < l_commitdate
|
and l_shipdate < l_commitdate
|
||||||
and l_receiptdate >= date '1994-01-01'
|
and CAST(l_receiptdate AS DATE) >= date '1994-01-01'
|
||||||
and l_receiptdate < date '1994-01-01' + interval '1' year
|
and CAST(l_receiptdate AS DATE) < date '1994-01-01' + interval '1' year
|
||||||
group by
|
group by
|
||||||
l_shipmode
|
l_shipmode
|
||||||
order by
|
order by
|
||||||
|
|
@ -813,8 +813,8 @@ from
|
||||||
part
|
part
|
||||||
where
|
where
|
||||||
l_partkey = p_partkey
|
l_partkey = p_partkey
|
||||||
and l_shipdate >= date '1995-09-01'
|
and CAST(l_shipdate AS DATE) >= date '1995-09-01'
|
||||||
and l_shipdate < date '1995-09-01' + interval '1' month;
|
and CAST(l_shipdate AS DATE) < date '1995-09-01' + interval '1' month;
|
||||||
SELECT
|
SELECT
|
||||||
100.00 * SUM(
|
100.00 * SUM(
|
||||||
CASE
|
CASE
|
||||||
|
|
@ -844,8 +844,8 @@ with revenue (supplier_no, total_revenue) as (
|
||||||
from
|
from
|
||||||
lineitem
|
lineitem
|
||||||
where
|
where
|
||||||
l_shipdate >= date '1996-01-01'
|
CAST(l_shipdate AS DATE) >= date '1996-01-01'
|
||||||
and l_shipdate < date '1996-01-01' + interval '3' month
|
and CAST(l_shipdate AS DATE) < date '1996-01-01' + interval '3' month
|
||||||
group by
|
group by
|
||||||
l_suppkey)
|
l_suppkey)
|
||||||
select
|
select
|
||||||
|
|
@ -1223,8 +1223,8 @@ where
|
||||||
where
|
where
|
||||||
l_partkey = ps_partkey
|
l_partkey = ps_partkey
|
||||||
and l_suppkey = ps_suppkey
|
and l_suppkey = ps_suppkey
|
||||||
and l_shipdate >= date '1994-01-01'
|
and CAST(l_shipdate AS DATE) >= date '1994-01-01'
|
||||||
and l_shipdate < date '1994-01-01' + interval '1' year
|
and CAST(l_shipdate AS DATE) < date '1994-01-01' + interval '1' year
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
and s_nationkey = n_nationkey
|
and s_nationkey = n_nationkey
|
||||||
|
|
|
||||||
|
|
@ -243,6 +243,15 @@ class TestExpressions(unittest.TestCase):
|
||||||
'SELECT * FROM a1 AS a /* a */, b.a /* b */, c.a2 /* c.a */, d2 /* d.a */ CROSS JOIN e.a CROSS JOIN "F" /* f-F.A */ CROSS JOIN g1.a /* g */',
|
'SELECT * FROM a1 AS a /* a */, b.a /* b */, c.a2 /* c.a */, d2 /* d.a */ CROSS JOIN e.a CROSS JOIN "F" /* f-F.A */ CROSS JOIN g1.a /* g */',
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
exp.replace_tables(
|
||||||
|
parse_one("select * from example.table", dialect="bigquery"),
|
||||||
|
{"example.table": "`my-project.example.table`"},
|
||||||
|
dialect="bigquery",
|
||||||
|
).sql(),
|
||||||
|
'SELECT * FROM "my-project".example.table /* example.table */',
|
||||||
|
)
|
||||||
|
|
||||||
def test_expand(self):
|
def test_expand(self):
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
exp.expand(
|
exp.expand(
|
||||||
|
|
|
||||||
|
|
@ -95,7 +95,7 @@ class TestLineage(unittest.TestCase):
|
||||||
downstream.source.sql(),
|
downstream.source.sql(),
|
||||||
"SELECT x.a AS a FROM x AS x",
|
"SELECT x.a AS a FROM x AS x",
|
||||||
)
|
)
|
||||||
self.assertEqual(downstream.alias, "")
|
self.assertEqual(downstream.alias, "z")
|
||||||
|
|
||||||
def test_lineage_source_with_star(self) -> None:
|
def test_lineage_source_with_star(self) -> None:
|
||||||
node = lineage(
|
node = lineage(
|
||||||
|
|
@ -153,7 +153,7 @@ class TestLineage(unittest.TestCase):
|
||||||
downstream = downstream.downstream[0]
|
downstream = downstream.downstream[0]
|
||||||
self.assertEqual(downstream.source.sql(), "(VALUES (1), (2)) AS t(a)")
|
self.assertEqual(downstream.source.sql(), "(VALUES (1), (2)) AS t(a)")
|
||||||
self.assertEqual(downstream.expression.sql(), "a")
|
self.assertEqual(downstream.expression.sql(), "a")
|
||||||
self.assertEqual(downstream.alias, "")
|
self.assertEqual(downstream.alias, "y")
|
||||||
|
|
||||||
def test_lineage_cte_name_appears_in_schema(self) -> None:
|
def test_lineage_cte_name_appears_in_schema(self) -> None:
|
||||||
schema = {"a": {"b": {"t1": {"c1": "int"}, "t2": {"c2": "int"}}}}
|
schema = {"a": {"b": {"t1": {"c1": "int"}, "t2": {"c2": "int"}}}}
|
||||||
|
|
@ -284,6 +284,35 @@ class TestLineage(unittest.TestCase):
|
||||||
self.assertEqual(downstream_b.name, "0")
|
self.assertEqual(downstream_b.name, "0")
|
||||||
self.assertEqual(downstream_b.source.sql(), "SELECT * FROM catalog.db.table_b AS table_b")
|
self.assertEqual(downstream_b.source.sql(), "SELECT * FROM catalog.db.table_b AS table_b")
|
||||||
|
|
||||||
|
def test_lineage_source_union(self) -> None:
|
||||||
|
query = "SELECT x, created_at FROM dataset;"
|
||||||
|
node = lineage(
|
||||||
|
"x",
|
||||||
|
query,
|
||||||
|
sources={
|
||||||
|
"dataset": """
|
||||||
|
SELECT *
|
||||||
|
FROM catalog.db.table_a
|
||||||
|
|
||||||
|
UNION
|
||||||
|
|
||||||
|
SELECT *
|
||||||
|
FROM catalog.db.table_b
|
||||||
|
"""
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(node.name, "x")
|
||||||
|
|
||||||
|
downstream_a = node.downstream[0]
|
||||||
|
self.assertEqual(downstream_a.name, "0")
|
||||||
|
self.assertEqual(downstream_a.alias, "dataset")
|
||||||
|
self.assertEqual(downstream_a.source.sql(), "SELECT * FROM catalog.db.table_a AS table_a")
|
||||||
|
downstream_b = node.downstream[1]
|
||||||
|
self.assertEqual(downstream_b.name, "0")
|
||||||
|
self.assertEqual(downstream_b.alias, "dataset")
|
||||||
|
self.assertEqual(downstream_b.source.sql(), "SELECT * FROM catalog.db.table_b AS table_b")
|
||||||
|
|
||||||
def test_select_star(self) -> None:
|
def test_select_star(self) -> None:
|
||||||
node = lineage("x", "SELECT x from (SELECT * from table_a)")
|
node = lineage("x", "SELECT x from (SELECT * from table_a)")
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -874,6 +874,12 @@ FROM READ_CSV('tests/fixtures/optimizer/tpc-h/nation.csv.gz', 'delimiter', '|')
|
||||||
self.assertEqual(case_ifs_expr.type.this, exp.DataType.Type.VARCHAR)
|
self.assertEqual(case_ifs_expr.type.this, exp.DataType.Type.VARCHAR)
|
||||||
self.assertEqual(case_ifs_expr.args["true"].type.this, exp.DataType.Type.VARCHAR)
|
self.assertEqual(case_ifs_expr.args["true"].type.this, exp.DataType.Type.VARCHAR)
|
||||||
|
|
||||||
|
timestamp = annotate_types(parse_one("TIMESTAMP(x)"))
|
||||||
|
self.assertEqual(timestamp.type.this, exp.DataType.Type.TIMESTAMP)
|
||||||
|
|
||||||
|
timestamptz = annotate_types(parse_one("TIMESTAMP(x)", read="bigquery"))
|
||||||
|
self.assertEqual(timestamptz.type.this, exp.DataType.Type.TIMESTAMPTZ)
|
||||||
|
|
||||||
def test_unknown_annotation(self):
|
def test_unknown_annotation(self):
|
||||||
schema = {"x": {"cola": "VARCHAR"}}
|
schema = {"x": {"cola": "VARCHAR"}}
|
||||||
sql = "SELECT x.cola + SOME_ANONYMOUS_FUNC(x.cola) AS col FROM x AS x"
|
sql = "SELECT x.cola + SOME_ANONYMOUS_FUNC(x.cola) AS col FROM x AS x"
|
||||||
|
|
|
||||||
|
|
@ -839,3 +839,16 @@ class TestParser(unittest.TestCase):
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
self.assertEqual(ast.sql(dialect=dialect), "CREATE SCHEMA catalog.schema")
|
self.assertEqual(ast.sql(dialect=dialect), "CREATE SCHEMA catalog.schema")
|
||||||
|
|
||||||
|
def test_values_as_identifier(self):
|
||||||
|
sql = "SELECT values FROM t WHERE values + 1 > x"
|
||||||
|
for dialect in (
|
||||||
|
"bigquery",
|
||||||
|
"clickhouse",
|
||||||
|
"duckdb",
|
||||||
|
"postgres",
|
||||||
|
"redshift",
|
||||||
|
"snowflake",
|
||||||
|
):
|
||||||
|
with self.subTest(dialect):
|
||||||
|
self.assertEqual(parse_one(sql, dialect=dialect).sql(dialect=dialect), sql)
|
||||||
|
|
|
||||||
|
|
@ -554,11 +554,12 @@ FROM base""",
|
||||||
self.validate(
|
self.validate(
|
||||||
"WITH A(filter) AS (VALUES 1, 2, 3) SELECT * FROM A WHERE filter >= 2",
|
"WITH A(filter) AS (VALUES 1, 2, 3) SELECT * FROM A WHERE filter >= 2",
|
||||||
"WITH A(filter) AS (VALUES (1), (2), (3)) SELECT * FROM A WHERE filter >= 2",
|
"WITH A(filter) AS (VALUES (1), (2), (3)) SELECT * FROM A WHERE filter >= 2",
|
||||||
|
read="presto",
|
||||||
)
|
)
|
||||||
self.validate(
|
self.validate(
|
||||||
"SELECT BOOL_OR(a > 10) FROM (VALUES 1, 2, 15) AS T(a)",
|
"SELECT BOOL_OR(a > 10) FROM (VALUES 1, 2, 15) AS T(a)",
|
||||||
"SELECT BOOL_OR(a > 10) FROM (VALUES (1), (2), (15)) AS T(a)",
|
"SELECT BOOL_OR(a > 10) FROM (VALUES (1), (2), (15)) AS T(a)",
|
||||||
write="presto",
|
read="presto",
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_alter(self):
|
def test_alter(self):
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue