Adding upstream version 18.2.0.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
9de781a59b
commit
ab14e550ff
GPG key ID:
FBB4F0E80A80222F
124 changed files with 60313 additions and 50346 deletions
File diff suppressed because one or more lines are too long
|
|
@ -96,7 +96,7 @@
|
|||
|
||||
<div class="docstring"><p><img src="sqlglot.svg" alt="SQLGlot logo" /></p>
|
||||
|
||||
<p>SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between <a href="https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py">19 different dialects</a> like <a href="https://duckdb.org/">DuckDB</a>, <a href="https://prestodb.io/">Presto</a>, <a href="https://spark.apache.org/">Spark</a>, <a href="https://www.snowflake.com/en/">Snowflake</a>, and <a href="https://cloud.google.com/bigquery/">BigQuery</a>. It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.</p>
|
||||
<p>SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between <a href="https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py">20 different dialects</a> like <a href="https://duckdb.org/">DuckDB</a>, <a href="https://prestodb.io/">Presto</a>, <a href="https://spark.apache.org/">Spark</a>, <a href="https://www.snowflake.com/en/">Snowflake</a>, and <a href="https://cloud.google.com/bigquery/">BigQuery</a>. It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.</p>
|
||||
|
||||
<p>It is a very comprehensive generic SQL parser with a robust <a href="https://github.com/tobymao/sqlglot/blob/main/tests/">test suite</a>. It is also quite <a href="#benchmarks">performant</a>, while being written purely in Python.</p>
|
||||
|
||||
|
|
@ -285,7 +285,7 @@
|
|||
|
||||
<h3 id="parser-errors">Parser Errors</h3>
|
||||
|
||||
<p>When the parser detects an error in the syntax, it raises a ParserError:</p>
|
||||
<p>When the parser detects an error in the syntax, it raises a ParseError:</p>
|
||||
|
||||
<div class="pdoc-code codehilite">
|
||||
<pre><span></span><code><span class="kn">import</span> <span class="nn">sqlglot</span>
|
||||
|
|
|
|||
|
|
@ -51,8 +51,8 @@
|
|||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos">1</span></a><span class="c1"># file generated by setuptools_scm</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a><span class="c1"># don't change, don't track in version control</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'17.11.0'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">17</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'18.1.0'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -96,11 +96,15 @@
|
|||
</ul></li>
|
||||
<li>See <a href="#registering-custom-schema-class">Registering Custom Schema</a> for information on how to skip this step if the information is stored externally.</li>
|
||||
</ul></li>
|
||||
<li>If your output SQL dialect is not Spark, then configure the SparkSession to use that dialect
|
||||
<ul>
|
||||
<li>Ex: <code>SparkSession().builder.config("sqlframe.dialect", "bigquery").getOrCreate()</code></li>
|
||||
<li>See <a href="https://github.com/tobymao/sqlglot/tree/main/sqlglot/dialects">dialects</a> for a full list of dialects.</li>
|
||||
</ul></li>
|
||||
<li>Add <code>.sql(pretty=True)</code> to your final DataFrame command to return a list of sql statements to run that command.
|
||||
<ul>
|
||||
<li>In most cases a single SQL statement is returned. Currently the only exception is when caching DataFrames which isn't supported in other dialects. </li>
|
||||
<li>Spark is the default output dialect. See <a href="https://github.com/tobymao/sqlglot/tree/main/sqlglot/dialects">dialects</a> for a full list of dialects.</li>
|
||||
<li>Ex: <code>.sql(pretty=True, dialect='bigquery')</code></li>
|
||||
<li>In most cases a single SQL statement is returned. Currently the only exception is when caching DataFrames which isn't supported in other dialects.</li>
|
||||
<li>Ex: <code>.sql(pretty=True)</code></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
|
||||
|
|
@ -111,6 +115,8 @@
|
|||
<span class="kn">from</span> <span class="nn">sqlglot.dataframe.sql.session</span> <span class="kn">import</span> <span class="n">SparkSession</span>
|
||||
<span class="kn">from</span> <span class="nn"><a href="dataframe/sql.html">sqlglot.dataframe.sql</a></span> <span class="kn">import</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">F</span>
|
||||
|
||||
<span class="n">dialect</span> <span class="o">=</span> <span class="s2">"spark"</span>
|
||||
|
||||
<span class="n">sqlglot</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
<span class="s1">'employee'</span><span class="p">,</span>
|
||||
<span class="p">{</span>
|
||||
|
|
@ -119,10 +125,10 @@
|
|||
<span class="s1">'lname'</span><span class="p">:</span> <span class="s1">'STRING'</span><span class="p">,</span>
|
||||
<span class="s1">'age'</span><span class="p">:</span> <span class="s1">'INT'</span><span class="p">,</span>
|
||||
<span class="p">},</span>
|
||||
<span class="n">dialect</span><span class="o">=</span><span class="s2">"spark"</span><span class="p">,</span>
|
||||
<span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span>
|
||||
<span class="p">)</span> <span class="c1"># Register the table structure prior to reading from the table</span>
|
||||
|
||||
<span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
||||
<span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="o">.</span><span class="n">builder</span><span class="o">.</span><span class="n">config</span><span class="p">(</span><span class="s2">"sqlframe.dialect"</span><span class="p">,</span> <span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">getOrCreate</span><span class="p">()</span>
|
||||
|
||||
<span class="n">df</span> <span class="o">=</span> <span class="p">(</span>
|
||||
<span class="n">spark</span>
|
||||
|
|
@ -131,7 +137,7 @@
|
|||
<span class="o">.</span><span class="n">agg</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">countDistinct</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"employee_id"</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"num_employees"</span><span class="p">))</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span> <span class="c1"># Spark will be the dialect used by default</span>
|
||||
<span class="nb">print</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</code></pre>
|
||||
</div>
|
||||
|
||||
|
|
@ -159,7 +165,7 @@ GROUP BY
|
|||
|
||||
<span class="n"><a href="schema.html">sqlglot.schema</a></span> <span class="o">=</span> <span class="n">ExternalSchema</span><span class="p">()</span>
|
||||
|
||||
<span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
||||
<span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span> <span class="c1"># Spark will be used by default is not specific in SparkSession config</span>
|
||||
|
||||
<span class="n">df</span> <span class="o">=</span> <span class="p">(</span>
|
||||
<span class="n">spark</span>
|
||||
|
|
@ -199,11 +205,14 @@ GROUP BY
|
|||
<span class="p">])</span>
|
||||
|
||||
<span class="n">sql_statements</span> <span class="o">=</span> <span class="p">(</span>
|
||||
<span class="n">SparkSession</span><span class="p">()</span>
|
||||
<span class="n">SparkSession</span>
|
||||
<span class="o">.</span><span class="n">builder</span>
|
||||
<span class="o">.</span><span class="n">config</span><span class="p">(</span><span class="s2">"sqlframe.dialect"</span><span class="p">,</span> <span class="s2">"bigquery"</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">getOrCreate</span><span class="p">()</span>
|
||||
<span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">groupBy</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"age"</span><span class="p">))</span>
|
||||
<span class="o">.</span><span class="n">agg</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">countDistinct</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"employee_id"</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"num_employees"</span><span class="p">))</span>
|
||||
<span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="s2">"bigquery"</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">sql</span><span class="p">()</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="n">result</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
|
|
@ -248,11 +257,14 @@ GROUP BY
|
|||
<span class="p">])</span>
|
||||
|
||||
<span class="n">sql_statements</span> <span class="o">=</span> <span class="p">(</span>
|
||||
<span class="n">SparkSession</span><span class="p">()</span>
|
||||
<span class="n">SparkSession</span>
|
||||
<span class="o">.</span><span class="n">builder</span>
|
||||
<span class="o">.</span><span class="n">config</span><span class="p">(</span><span class="s2">"sqlframe.dialect"</span><span class="p">,</span> <span class="s2">"snowflake"</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">getOrCreate</span><span class="p">()</span>
|
||||
<span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">groupBy</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"age"</span><span class="p">))</span>
|
||||
<span class="o">.</span><span class="n">agg</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">countDistinct</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"lname"</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"num_employees"</span><span class="p">))</span>
|
||||
<span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="s2">"snowflake"</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">sql</span><span class="p">()</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="k">try</span><span class="p">:</span>
|
||||
|
|
@ -294,7 +306,7 @@ GROUP BY
|
|||
<span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">groupBy</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"age"</span><span class="p">))</span>
|
||||
<span class="o">.</span><span class="n">agg</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">countDistinct</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">col</span><span class="p">(</span><span class="s2">"employee_id"</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"num_employees"</span><span class="p">))</span>
|
||||
<span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="s2">"spark"</span><span class="p">)</span>
|
||||
<span class="o">.</span><span class="n">sql</span><span class="p">()</span>
|
||||
<span class="p">)</span>
|
||||
|
||||
<span class="n">pyspark</span> <span class="o">=</span> <span class="n">PySparkSession</span><span class="o">.</span><span class="n">builder</span><span class="o">.</span><span class="n">master</span><span class="p">(</span><span class="s2">"local[*]"</span><span class="p">)</span><span class="o">.</span><span class="n">getOrCreate</span><span class="p">()</span>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -38,6 +38,7 @@
|
|||
<li><a href="dialects/clickhouse.html">clickhouse</a></li>
|
||||
<li><a href="dialects/databricks.html">databricks</a></li>
|
||||
<li><a href="dialects/dialect.html">dialect</a></li>
|
||||
<li><a href="dialects/doris.html">doris</a></li>
|
||||
<li><a href="dialects/drill.html">drill</a></li>
|
||||
<li><a href="dialects/duckdb.html">duckdb</a></li>
|
||||
<li><a href="dialects/hive.html">hive</a></li>
|
||||
|
|
@ -195,23 +196,24 @@ logic for some expressions; this is usually done by adding new entries to the <c
|
|||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.clickhouse</span> <span class="kn">import</span> <span class="n">ClickHouse</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.databricks</span> <span class="kn">import</span> <span class="n">Databricks</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">Dialects</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.drill</span> <span class="kn">import</span> <span class="n">Drill</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.duckdb</span> <span class="kn">import</span> <span class="n">DuckDB</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.hive</span> <span class="kn">import</span> <span class="n">Hive</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.mysql</span> <span class="kn">import</span> <span class="n">MySQL</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.oracle</span> <span class="kn">import</span> <span class="n">Oracle</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.postgres</span> <span class="kn">import</span> <span class="n">Postgres</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.presto</span> <span class="kn">import</span> <span class="n">Presto</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.redshift</span> <span class="kn">import</span> <span class="n">Redshift</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.snowflake</span> <span class="kn">import</span> <span class="n">Snowflake</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.spark</span> <span class="kn">import</span> <span class="n">Spark</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.spark2</span> <span class="kn">import</span> <span class="n">Spark2</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.sqlite</span> <span class="kn">import</span> <span class="n">SQLite</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.starrocks</span> <span class="kn">import</span> <span class="n">StarRocks</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.tableau</span> <span class="kn">import</span> <span class="n">Tableau</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.teradata</span> <span class="kn">import</span> <span class="n">Teradata</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.trino</span> <span class="kn">import</span> <span class="n">Trino</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.tsql</span> <span class="kn">import</span> <span class="n">TSQL</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.doris</span> <span class="kn">import</span> <span class="n">Doris</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.drill</span> <span class="kn">import</span> <span class="n">Drill</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.duckdb</span> <span class="kn">import</span> <span class="n">DuckDB</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.hive</span> <span class="kn">import</span> <span class="n">Hive</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.mysql</span> <span class="kn">import</span> <span class="n">MySQL</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.oracle</span> <span class="kn">import</span> <span class="n">Oracle</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.postgres</span> <span class="kn">import</span> <span class="n">Postgres</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.presto</span> <span class="kn">import</span> <span class="n">Presto</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.redshift</span> <span class="kn">import</span> <span class="n">Redshift</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.snowflake</span> <span class="kn">import</span> <span class="n">Snowflake</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.spark</span> <span class="kn">import</span> <span class="n">Spark</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.spark2</span> <span class="kn">import</span> <span class="n">Spark2</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.sqlite</span> <span class="kn">import</span> <span class="n">SQLite</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.starrocks</span> <span class="kn">import</span> <span class="n">StarRocks</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.tableau</span> <span class="kn">import</span> <span class="n">Tableau</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.teradata</span> <span class="kn">import</span> <span class="n">Teradata</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.trino</span> <span class="kn">import</span> <span class="n">Trino</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.tsql</span> <span class="kn">import</span> <span class="n">TSQL</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
1705
docs/sqlglot/dialects/doris.html
Normal file
1705
docs/sqlglot/dialects/doris.html
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -58,6 +58,9 @@
|
|||
<li>
|
||||
<a class="variable" href="#logger">logger</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="variable" href="#PYTHON_TYPE_TO_SQLGLOT">PYTHON_TYPE_TO_SQLGLOT</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#execute">execute</a>
|
||||
</li>
|
||||
|
|
@ -362,62 +365,68 @@
|
|||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a><span class="k">def</span> <span class="nf">execute</span><span class="p">(</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a> <span class="n">tables</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a><span class="sd"> Run a sql query against data.</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> sql: a sql statement.</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> schema: database schema.</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a><span class="sd"> This can either be an instance of `Schema` or a mapping in one of the following forms:</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> 1. {table: {col: type}}</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="sd"> 2. {db: {table: {col: type}}}</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> 3. {catalog: {db: {table: {col: type}}}}</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. "spark", "hive", "presto", "mysql").</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> tables: additional tables to register.</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a><span class="sd"> Simple columnar data structure.</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="n">tables_</span> <span class="o">=</span> <span class="n">ensure_tables</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a><span class="n">PYTHON_TYPE_TO_SQLGLOT</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="s2">"dict"</span><span class="p">:</span> <span class="s2">"MAP"</span><span class="p">,</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a><span class="p">}</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a><span class="k">def</span> <span class="nf">execute</span><span class="p">(</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a> <span class="n">tables</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a><span class="sd"> Run a sql query against data.</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> sql: a sql statement.</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a><span class="sd"> schema: database schema.</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> This can either be an instance of `Schema` or a mapping in one of the following forms:</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> 1. {table: {col: type}}</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> 2. {db: {table: {col: type}}}</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a><span class="sd"> 3. {catalog: {db: {table: {col: type}}}}</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. "spark", "hive", "presto", "mysql").</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="sd"> tables: additional tables to register.</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">schema</span><span class="p">:</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="n">flattened_tables</span> <span class="o">=</span> <span class="n">flatten_schema</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="n">depth</span><span class="o">=</span><span class="n">dict_depth</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">))</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="k">for</span> <span class="n">keys</span> <span class="ow">in</span> <span class="n">flattened_tables</span><span class="p">:</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">nested_get</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="o">*</span><span class="nb">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">keys</span><span class="p">))</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="k">assert</span> <span class="n">table</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="nb">type</span><span class="p">(</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="vm">__name__</span><span class="p">)</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a><span class="sd"> Simple columnar data structure.</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">tables_</span> <span class="o">=</span> <span class="n">ensure_tables</span><span class="p">(</span><span class="n">tables</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">schema</span><span class="p">:</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="n">flattened_tables</span> <span class="o">=</span> <span class="n">flatten_schema</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="n">depth</span><span class="o">=</span><span class="n">dict_depth</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">))</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="k">for</span> <span class="n">keys</span> <span class="ow">in</span> <span class="n">flattened_tables</span><span class="p">:</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">nested_get</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="o">*</span><span class="nb">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">keys</span><span class="p">))</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a> <span class="k">assert</span> <span class="n">table</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">"Tables must support the same table args as schema"</span><span class="p">)</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="n">py_type</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="vm">__name__</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="n">PYTHON_TYPE_TO_SQLGLOT</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">py_type</span><span class="p">)</span> <span class="ow">or</span> <span class="n">py_type</span><span class="p">)</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">"Tables must support the same table args as schema"</span><span class="p">)</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -433,6 +442,18 @@
|
|||
|
||||
|
||||
|
||||
</section>
|
||||
<section id="PYTHON_TYPE_TO_SQLGLOT">
|
||||
<div class="attr variable">
|
||||
<span class="name">PYTHON_TYPE_TO_SQLGLOT</span> =
|
||||
<span class="default_value">{'dict': 'MAP'}</span>
|
||||
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#PYTHON_TYPE_TO_SQLGLOT"></a>
|
||||
|
||||
|
||||
|
||||
</section>
|
||||
<section id="execute">
|
||||
<input id="execute-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
|
|
@ -445,62 +466,63 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#execute"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="execute-32"><a href="#execute-32"><span class="linenos">32</span></a><span class="k">def</span> <span class="nf">execute</span><span class="p">(</span>
|
||||
</span><span id="execute-33"><a href="#execute-33"><span class="linenos">33</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="execute-34"><a href="#execute-34"><span class="linenos">34</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="execute-35"><a href="#execute-35"><span class="linenos">35</span></a> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="execute-36"><a href="#execute-36"><span class="linenos">36</span></a> <span class="n">tables</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="execute-37"><a href="#execute-37"><span class="linenos">37</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="execute-38"><a href="#execute-38"><span class="linenos">38</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="execute-39"><a href="#execute-39"><span class="linenos">39</span></a><span class="sd"> Run a sql query against data.</span>
|
||||
</span><span id="execute-40"><a href="#execute-40"><span class="linenos">40</span></a>
|
||||
</span><span id="execute-41"><a href="#execute-41"><span class="linenos">41</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="execute-42"><a href="#execute-42"><span class="linenos">42</span></a><span class="sd"> sql: a sql statement.</span>
|
||||
</span><span id="execute-43"><a href="#execute-43"><span class="linenos">43</span></a><span class="sd"> schema: database schema.</span>
|
||||
</span><span id="execute-44"><a href="#execute-44"><span class="linenos">44</span></a><span class="sd"> This can either be an instance of `Schema` or a mapping in one of the following forms:</span>
|
||||
</span><span id="execute-45"><a href="#execute-45"><span class="linenos">45</span></a><span class="sd"> 1. {table: {col: type}}</span>
|
||||
</span><span id="execute-46"><a href="#execute-46"><span class="linenos">46</span></a><span class="sd"> 2. {db: {table: {col: type}}}</span>
|
||||
</span><span id="execute-47"><a href="#execute-47"><span class="linenos">47</span></a><span class="sd"> 3. {catalog: {db: {table: {col: type}}}}</span>
|
||||
</span><span id="execute-48"><a href="#execute-48"><span class="linenos">48</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. "spark", "hive", "presto", "mysql").</span>
|
||||
</span><span id="execute-49"><a href="#execute-49"><span class="linenos">49</span></a><span class="sd"> tables: additional tables to register.</span>
|
||||
</span><span id="execute-50"><a href="#execute-50"><span class="linenos">50</span></a>
|
||||
</span><span id="execute-51"><a href="#execute-51"><span class="linenos">51</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="execute-52"><a href="#execute-52"><span class="linenos">52</span></a><span class="sd"> Simple columnar data structure.</span>
|
||||
</span><span id="execute-53"><a href="#execute-53"><span class="linenos">53</span></a><span class="sd"> """</span>
|
||||
</span><span id="execute-54"><a href="#execute-54"><span class="linenos">54</span></a> <span class="n">tables_</span> <span class="o">=</span> <span class="n">ensure_tables</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="execute-37"><a href="#execute-37"><span class="linenos">37</span></a><span class="k">def</span> <span class="nf">execute</span><span class="p">(</span>
|
||||
</span><span id="execute-38"><a href="#execute-38"><span class="linenos">38</span></a> <span class="n">sql</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">Expression</span><span class="p">,</span>
|
||||
</span><span id="execute-39"><a href="#execute-39"><span class="linenos">39</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="execute-40"><a href="#execute-40"><span class="linenos">40</span></a> <span class="n">read</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="execute-41"><a href="#execute-41"><span class="linenos">41</span></a> <span class="n">tables</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="execute-42"><a href="#execute-42"><span class="linenos">42</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="execute-43"><a href="#execute-43"><span class="linenos">43</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="execute-44"><a href="#execute-44"><span class="linenos">44</span></a><span class="sd"> Run a sql query against data.</span>
|
||||
</span><span id="execute-45"><a href="#execute-45"><span class="linenos">45</span></a>
|
||||
</span><span id="execute-46"><a href="#execute-46"><span class="linenos">46</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="execute-47"><a href="#execute-47"><span class="linenos">47</span></a><span class="sd"> sql: a sql statement.</span>
|
||||
</span><span id="execute-48"><a href="#execute-48"><span class="linenos">48</span></a><span class="sd"> schema: database schema.</span>
|
||||
</span><span id="execute-49"><a href="#execute-49"><span class="linenos">49</span></a><span class="sd"> This can either be an instance of `Schema` or a mapping in one of the following forms:</span>
|
||||
</span><span id="execute-50"><a href="#execute-50"><span class="linenos">50</span></a><span class="sd"> 1. {table: {col: type}}</span>
|
||||
</span><span id="execute-51"><a href="#execute-51"><span class="linenos">51</span></a><span class="sd"> 2. {db: {table: {col: type}}}</span>
|
||||
</span><span id="execute-52"><a href="#execute-52"><span class="linenos">52</span></a><span class="sd"> 3. {catalog: {db: {table: {col: type}}}}</span>
|
||||
</span><span id="execute-53"><a href="#execute-53"><span class="linenos">53</span></a><span class="sd"> read: the SQL dialect to apply during parsing (eg. "spark", "hive", "presto", "mysql").</span>
|
||||
</span><span id="execute-54"><a href="#execute-54"><span class="linenos">54</span></a><span class="sd"> tables: additional tables to register.</span>
|
||||
</span><span id="execute-55"><a href="#execute-55"><span class="linenos">55</span></a>
|
||||
</span><span id="execute-56"><a href="#execute-56"><span class="linenos">56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">schema</span><span class="p">:</span>
|
||||
</span><span id="execute-57"><a href="#execute-57"><span class="linenos">57</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="execute-58"><a href="#execute-58"><span class="linenos">58</span></a> <span class="n">flattened_tables</span> <span class="o">=</span> <span class="n">flatten_schema</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="n">depth</span><span class="o">=</span><span class="n">dict_depth</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">))</span>
|
||||
</span><span id="execute-59"><a href="#execute-59"><span class="linenos">59</span></a>
|
||||
</span><span id="execute-60"><a href="#execute-60"><span class="linenos">60</span></a> <span class="k">for</span> <span class="n">keys</span> <span class="ow">in</span> <span class="n">flattened_tables</span><span class="p">:</span>
|
||||
</span><span id="execute-61"><a href="#execute-61"><span class="linenos">61</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">nested_get</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="o">*</span><span class="nb">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">keys</span><span class="p">))</span>
|
||||
</span><span id="execute-62"><a href="#execute-62"><span class="linenos">62</span></a> <span class="k">assert</span> <span class="n">table</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||
</span><span id="execute-63"><a href="#execute-63"><span class="linenos">63</span></a>
|
||||
</span><span id="execute-64"><a href="#execute-64"><span class="linenos">64</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="execute-65"><a href="#execute-65"><span class="linenos">65</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="nb">type</span><span class="p">(</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="vm">__name__</span><span class="p">)</span>
|
||||
</span><span id="execute-66"><a href="#execute-66"><span class="linenos">66</span></a>
|
||||
</span><span id="execute-67"><a href="#execute-67"><span class="linenos">67</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="execute-56"><a href="#execute-56"><span class="linenos">56</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="execute-57"><a href="#execute-57"><span class="linenos">57</span></a><span class="sd"> Simple columnar data structure.</span>
|
||||
</span><span id="execute-58"><a href="#execute-58"><span class="linenos">58</span></a><span class="sd"> """</span>
|
||||
</span><span id="execute-59"><a href="#execute-59"><span class="linenos">59</span></a> <span class="n">tables_</span> <span class="o">=</span> <span class="n">ensure_tables</span><span class="p">(</span><span class="n">tables</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="execute-60"><a href="#execute-60"><span class="linenos">60</span></a>
|
||||
</span><span id="execute-61"><a href="#execute-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">schema</span><span class="p">:</span>
|
||||
</span><span id="execute-62"><a href="#execute-62"><span class="linenos">62</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="execute-63"><a href="#execute-63"><span class="linenos">63</span></a> <span class="n">flattened_tables</span> <span class="o">=</span> <span class="n">flatten_schema</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="n">depth</span><span class="o">=</span><span class="n">dict_depth</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">))</span>
|
||||
</span><span id="execute-64"><a href="#execute-64"><span class="linenos">64</span></a>
|
||||
</span><span id="execute-65"><a href="#execute-65"><span class="linenos">65</span></a> <span class="k">for</span> <span class="n">keys</span> <span class="ow">in</span> <span class="n">flattened_tables</span><span class="p">:</span>
|
||||
</span><span id="execute-66"><a href="#execute-66"><span class="linenos">66</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">nested_get</span><span class="p">(</span><span class="n">tables_</span><span class="o">.</span><span class="n">mapping</span><span class="p">,</span> <span class="o">*</span><span class="nb">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">keys</span><span class="p">))</span>
|
||||
</span><span id="execute-67"><a href="#execute-67"><span class="linenos">67</span></a> <span class="k">assert</span> <span class="n">table</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||
</span><span id="execute-68"><a href="#execute-68"><span class="linenos">68</span></a>
|
||||
</span><span id="execute-69"><a href="#execute-69"><span class="linenos">69</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span>
|
||||
</span><span id="execute-70"><a href="#execute-70"><span class="linenos">70</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">"Tables must support the same table args as schema"</span><span class="p">)</span>
|
||||
</span><span id="execute-71"><a href="#execute-71"><span class="linenos">71</span></a>
|
||||
</span><span id="execute-72"><a href="#execute-72"><span class="linenos">72</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-73"><a href="#execute-73"><span class="linenos">73</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="execute-69"><a href="#execute-69"><span class="linenos">69</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="execute-70"><a href="#execute-70"><span class="linenos">70</span></a> <span class="n">py_type</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="vm">__name__</span>
|
||||
</span><span id="execute-71"><a href="#execute-71"><span class="linenos">71</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="n">PYTHON_TYPE_TO_SQLGLOT</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">py_type</span><span class="p">)</span> <span class="ow">or</span> <span class="n">py_type</span><span class="p">)</span>
|
||||
</span><span id="execute-72"><a href="#execute-72"><span class="linenos">72</span></a>
|
||||
</span><span id="execute-73"><a href="#execute-73"><span class="linenos">73</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="execute-74"><a href="#execute-74"><span class="linenos">74</span></a>
|
||||
</span><span id="execute-75"><a href="#execute-75"><span class="linenos">75</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-76"><a href="#execute-76"><span class="linenos">76</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="execute-75"><a href="#execute-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span>
|
||||
</span><span id="execute-76"><a href="#execute-76"><span class="linenos">76</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">"Tables must support the same table args as schema"</span><span class="p">)</span>
|
||||
</span><span id="execute-77"><a href="#execute-77"><span class="linenos">77</span></a>
|
||||
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a>
|
||||
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a>
|
||||
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-84"><a href="#execute-84"><span class="linenos">84</span></a>
|
||||
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-86"><a href="#execute-86"><span class="linenos">86</span></a>
|
||||
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
|
||||
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a>
|
||||
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimization finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Optimized SQL: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
|
||||
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a>
|
||||
</span><span id="execute-84"><a href="#execute-84"><span class="linenos">84</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a>
|
||||
</span><span id="execute-86"><a href="#execute-86"><span class="linenos">86</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Logical Plan: </span><span class="si">%s</span><span class="s2">"</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a>
|
||||
</span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
|
||||
</span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
|
||||
</span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a>
|
||||
</span><span id="execute-91"><a href="#execute-91"><span class="linenos">91</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Query finished: </span><span class="si">%f</span><span class="s2">"</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
|
||||
</span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a>
|
||||
</span><span id="execute-93"><a href="#execute-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -147,135 +147,147 @@
|
|||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">dict_depth</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">AbstractMappingSchema</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">DialectType</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">dict_depth</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">AbstractMappingSchema</span><span class="p">,</span> <span class="n">normalize_name</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="k">class</span> <span class="nc">Table</span><span class="p">:</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="n">column_range</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span> <span class="o">=</span> <span class="n">rows</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a> <span class="k">if</span> <span class="n">rows</span><span class="p">:</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range_reader</span> <span class="o">=</span> <span class="n">RangeReader</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="k">def</span> <span class="nf">add_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">columns</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">+=</span> <span class="n">columns</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">:</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">start</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">stop</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="p">)</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">row</span><span class="p">):</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="nd">@property</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="k">def</span> <span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">def</span> <span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">column</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="p">)</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">></span> <span class="mi">10</span><span class="p">:</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">break</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="p">)</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="p">)</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">return</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="k">class</span> <span class="nc">Table</span><span class="p">:</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="n">column_range</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span> <span class="o">=</span> <span class="n">rows</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a> <span class="k">if</span> <span class="n">rows</span><span class="p">:</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range_reader</span> <span class="o">=</span> <span class="n">RangeReader</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="k">def</span> <span class="nf">add_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">columns</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">+=</span> <span class="n">columns</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">:</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">start</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">stop</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="p">)</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">row</span><span class="p">):</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="nd">@property</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">def</span> <span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="k">def</span> <span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">column</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="p">)</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">></span> <span class="mi">10</span><span class="p">:</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">break</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="p">)</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="p">)</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">return</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="k">class</span> <span class="nc">TableIter</span><span class="p">:</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">def</span> <span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o"><</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="k">class</span> <span class="nc">TableIter</span><span class="p">:</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">def</span> <span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o"><</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="k">class</span> <span class="nc">RangeReader</span><span class="p">:</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="k">class</span> <span class="nc">RangeReader</span><span class="p">:</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a><span class="k">class</span> <span class="nc">RowReader</span><span class="p">:</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="p">}</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a><span class="k">class</span> <span class="nc">RowReader</span><span class="p">:</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="p">}</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a><span class="k">class</span> <span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">[</span><span class="n">Table</span><span class="p">]):</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">pass</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a><span class="k">class</span> <span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">[</span><span class="n">Table</span><span class="p">]):</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">pass</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a><span class="k">def</span> <span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">])</span> <span class="o">-></span> <span class="n">Tables</span><span class="p">:</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">))</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a><span class="k">def</span> <span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tables</span><span class="p">:</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="k">def</span> <span class="nf">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">d</span><span class="p">:</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">depth</span> <span class="o">=</span> <span class="n">dict_depth</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a><span class="k">def</span> <span class="nf">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">d</span><span class="p">:</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">depth</span> <span class="o">=</span> <span class="n">dict_depth</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">></span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="n">_ensure_tables</span><span class="p">(</span><span class="n">v</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">result</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">table</span> <span class="k">else</span> <span class="p">()</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">rows</span> <span class="o">=</span> <span class="p">[</span><span class="nb">tuple</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">c</span><span class="p">]</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="p">]</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">result</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="n">rows</span><span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span> <span class="n">_ensure_tables</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="p">}</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">normalize_name</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">table</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="p">{</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">):</span> <span class="n">value</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">row</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="p">}</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">]</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">column_name</span> <span class="k">for</span> <span class="n">column_name</span> <span class="ow">in</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">table</span> <span class="k">else</span> <span class="p">()</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">rows</span> <span class="o">=</span> <span class="p">[</span><span class="nb">tuple</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">column_names</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="p">]</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">column_names</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="n">rows</span><span class="p">)</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">result</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -291,64 +303,64 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Table"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table-10"><a href="#Table-10"><span class="linenos">10</span></a><span class="k">class</span> <span class="nc">Table</span><span class="p">:</span>
|
||||
</span><span id="Table-11"><a href="#Table-11"><span class="linenos">11</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="Table-12"><a href="#Table-12"><span class="linenos">12</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-13"><a href="#Table-13"><span class="linenos">13</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="n">column_range</span>
|
||||
</span><span id="Table-14"><a href="#Table-14"><span class="linenos">14</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="Table-15"><a href="#Table-15"><span class="linenos">15</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span> <span class="o">=</span> <span class="n">rows</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="Table-16"><a href="#Table-16"><span class="linenos">16</span></a> <span class="k">if</span> <span class="n">rows</span><span class="p">:</span>
|
||||
</span><span id="Table-17"><a href="#Table-17"><span class="linenos">17</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-18"><a href="#Table-18"><span class="linenos">18</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range_reader</span> <span class="o">=</span> <span class="n">RangeReader</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="Table-19"><a href="#Table-19"><span class="linenos">19</span></a>
|
||||
</span><span id="Table-20"><a href="#Table-20"><span class="linenos">20</span></a> <span class="k">def</span> <span class="nf">add_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">columns</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Table-21"><a href="#Table-21"><span class="linenos">21</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">+=</span> <span class="n">columns</span>
|
||||
</span><span id="Table-22"><a href="#Table-22"><span class="linenos">22</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">:</span>
|
||||
</span><span id="Table-23"><a href="#Table-23"><span class="linenos">23</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span>
|
||||
</span><span id="Table-24"><a href="#Table-24"><span class="linenos">24</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">start</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">stop</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-25"><a href="#Table-25"><span class="linenos">25</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-26"><a href="#Table-26"><span class="linenos">26</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="Table-27"><a href="#Table-27"><span class="linenos">27</span></a>
|
||||
</span><span id="Table-28"><a href="#Table-28"><span class="linenos">28</span></a> <span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">row</span><span class="p">):</span>
|
||||
</span><span id="Table-29"><a href="#Table-29"><span class="linenos">29</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-30"><a href="#Table-30"><span class="linenos">30</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
|
||||
</span><span id="Table-31"><a href="#Table-31"><span class="linenos">31</span></a>
|
||||
</span><span id="Table-32"><a href="#Table-32"><span class="linenos">32</span></a> <span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-33"><a href="#Table-33"><span class="linenos">33</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="Table-34"><a href="#Table-34"><span class="linenos">34</span></a>
|
||||
</span><span id="Table-35"><a href="#Table-35"><span class="linenos">35</span></a> <span class="nd">@property</span>
|
||||
</span><span id="Table-36"><a href="#Table-36"><span class="linenos">36</span></a> <span class="k">def</span> <span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-37"><a href="#Table-37"><span class="linenos">37</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-38"><a href="#Table-38"><span class="linenos">38</span></a>
|
||||
</span><span id="Table-39"><a href="#Table-39"><span class="linenos">39</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-40"><a href="#Table-40"><span class="linenos">40</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
|
||||
</span><span id="Table-41"><a href="#Table-41"><span class="linenos">41</span></a>
|
||||
</span><span id="Table-42"><a href="#Table-42"><span class="linenos">42</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-43"><a href="#Table-43"><span class="linenos">43</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="Table-44"><a href="#Table-44"><span class="linenos">44</span></a>
|
||||
</span><span id="Table-45"><a href="#Table-45"><span class="linenos">45</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
|
||||
</span><span id="Table-46"><a href="#Table-46"><span class="linenos">46</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="Table-47"><a href="#Table-47"><span class="linenos">47</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
|
||||
</span><span id="Table-48"><a href="#Table-48"><span class="linenos">48</span></a>
|
||||
</span><span id="Table-49"><a href="#Table-49"><span class="linenos">49</span></a> <span class="k">def</span> <span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-50"><a href="#Table-50"><span class="linenos">50</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="Table-51"><a href="#Table-51"><span class="linenos">51</span></a> <span class="n">column</span>
|
||||
</span><span id="Table-52"><a href="#Table-52"><span class="linenos">52</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-53"><a href="#Table-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
|
||||
</span><span id="Table-54"><a href="#Table-54"><span class="linenos">54</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-55"><a href="#Table-55"><span class="linenos">55</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
|
||||
</span><span id="Table-56"><a href="#Table-56"><span class="linenos">56</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
|
||||
</span><span id="Table-57"><a href="#Table-57"><span class="linenos">57</span></a>
|
||||
</span><span id="Table-58"><a href="#Table-58"><span class="linenos">58</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-59"><a href="#Table-59"><span class="linenos">59</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">></span> <span class="mi">10</span><span class="p">:</span>
|
||||
</span><span id="Table-60"><a href="#Table-60"><span class="linenos">60</span></a> <span class="k">break</span>
|
||||
</span><span id="Table-61"><a href="#Table-61"><span class="linenos">61</span></a>
|
||||
</span><span id="Table-62"><a href="#Table-62"><span class="linenos">62</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="Table-63"><a href="#Table-63"><span class="linenos">63</span></a> <span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
|
||||
</span><span id="Table-64"><a href="#Table-64"><span class="linenos">64</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Table-65"><a href="#Table-65"><span class="linenos">65</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-66"><a href="#Table-66"><span class="linenos">66</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-67"><a href="#Table-67"><span class="linenos">67</span></a> <span class="k">return</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table-11"><a href="#Table-11"><span class="linenos">11</span></a><span class="k">class</span> <span class="nc">Table</span><span class="p">:</span>
|
||||
</span><span id="Table-12"><a href="#Table-12"><span class="linenos">12</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="Table-13"><a href="#Table-13"><span class="linenos">13</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-14"><a href="#Table-14"><span class="linenos">14</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="n">column_range</span>
|
||||
</span><span id="Table-15"><a href="#Table-15"><span class="linenos">15</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="Table-16"><a href="#Table-16"><span class="linenos">16</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span> <span class="o">=</span> <span class="n">rows</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="Table-17"><a href="#Table-17"><span class="linenos">17</span></a> <span class="k">if</span> <span class="n">rows</span><span class="p">:</span>
|
||||
</span><span id="Table-18"><a href="#Table-18"><span class="linenos">18</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-19"><a href="#Table-19"><span class="linenos">19</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range_reader</span> <span class="o">=</span> <span class="n">RangeReader</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="Table-20"><a href="#Table-20"><span class="linenos">20</span></a>
|
||||
</span><span id="Table-21"><a href="#Table-21"><span class="linenos">21</span></a> <span class="k">def</span> <span class="nf">add_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">columns</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Table-22"><a href="#Table-22"><span class="linenos">22</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">+=</span> <span class="n">columns</span>
|
||||
</span><span id="Table-23"><a href="#Table-23"><span class="linenos">23</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">:</span>
|
||||
</span><span id="Table-24"><a href="#Table-24"><span class="linenos">24</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span>
|
||||
</span><span id="Table-25"><a href="#Table-25"><span class="linenos">25</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">start</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">stop</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-26"><a href="#Table-26"><span class="linenos">26</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-27"><a href="#Table-27"><span class="linenos">27</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="Table-28"><a href="#Table-28"><span class="linenos">28</span></a>
|
||||
</span><span id="Table-29"><a href="#Table-29"><span class="linenos">29</span></a> <span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">row</span><span class="p">):</span>
|
||||
</span><span id="Table-30"><a href="#Table-30"><span class="linenos">30</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-31"><a href="#Table-31"><span class="linenos">31</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
|
||||
</span><span id="Table-32"><a href="#Table-32"><span class="linenos">32</span></a>
|
||||
</span><span id="Table-33"><a href="#Table-33"><span class="linenos">33</span></a> <span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-34"><a href="#Table-34"><span class="linenos">34</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="Table-35"><a href="#Table-35"><span class="linenos">35</span></a>
|
||||
</span><span id="Table-36"><a href="#Table-36"><span class="linenos">36</span></a> <span class="nd">@property</span>
|
||||
</span><span id="Table-37"><a href="#Table-37"><span class="linenos">37</span></a> <span class="k">def</span> <span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-38"><a href="#Table-38"><span class="linenos">38</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-39"><a href="#Table-39"><span class="linenos">39</span></a>
|
||||
</span><span id="Table-40"><a href="#Table-40"><span class="linenos">40</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-41"><a href="#Table-41"><span class="linenos">41</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
|
||||
</span><span id="Table-42"><a href="#Table-42"><span class="linenos">42</span></a>
|
||||
</span><span id="Table-43"><a href="#Table-43"><span class="linenos">43</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-44"><a href="#Table-44"><span class="linenos">44</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span><span id="Table-45"><a href="#Table-45"><span class="linenos">45</span></a>
|
||||
</span><span id="Table-46"><a href="#Table-46"><span class="linenos">46</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
|
||||
</span><span id="Table-47"><a href="#Table-47"><span class="linenos">47</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="Table-48"><a href="#Table-48"><span class="linenos">48</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
|
||||
</span><span id="Table-49"><a href="#Table-49"><span class="linenos">49</span></a>
|
||||
</span><span id="Table-50"><a href="#Table-50"><span class="linenos">50</span></a> <span class="k">def</span> <span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-51"><a href="#Table-51"><span class="linenos">51</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="Table-52"><a href="#Table-52"><span class="linenos">52</span></a> <span class="n">column</span>
|
||||
</span><span id="Table-53"><a href="#Table-53"><span class="linenos">53</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table-54"><a href="#Table-54"><span class="linenos">54</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
|
||||
</span><span id="Table-55"><a href="#Table-55"><span class="linenos">55</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-56"><a href="#Table-56"><span class="linenos">56</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
|
||||
</span><span id="Table-57"><a href="#Table-57"><span class="linenos">57</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
|
||||
</span><span id="Table-58"><a href="#Table-58"><span class="linenos">58</span></a>
|
||||
</span><span id="Table-59"><a href="#Table-59"><span class="linenos">59</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table-60"><a href="#Table-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">></span> <span class="mi">10</span><span class="p">:</span>
|
||||
</span><span id="Table-61"><a href="#Table-61"><span class="linenos">61</span></a> <span class="k">break</span>
|
||||
</span><span id="Table-62"><a href="#Table-62"><span class="linenos">62</span></a>
|
||||
</span><span id="Table-63"><a href="#Table-63"><span class="linenos">63</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="Table-64"><a href="#Table-64"><span class="linenos">64</span></a> <span class="s2">" "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
|
||||
</span><span id="Table-65"><a href="#Table-65"><span class="linenos">65</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Table-66"><a href="#Table-66"><span class="linenos">66</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-67"><a href="#Table-67"><span class="linenos">67</span></a> <span class="p">)</span>
|
||||
</span><span id="Table-68"><a href="#Table-68"><span class="linenos">68</span></a> <span class="k">return</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -364,14 +376,14 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Table.__init__"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.__init__-11"><a href="#Table.__init__-11"><span class="linenos">11</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="Table.__init__-12"><a href="#Table.__init__-12"><span class="linenos">12</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.__init__-13"><a href="#Table.__init__-13"><span class="linenos">13</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="n">column_range</span>
|
||||
</span><span id="Table.__init__-14"><a href="#Table.__init__-14"><span class="linenos">14</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="Table.__init__-15"><a href="#Table.__init__-15"><span class="linenos">15</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span> <span class="o">=</span> <span class="n">rows</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="Table.__init__-16"><a href="#Table.__init__-16"><span class="linenos">16</span></a> <span class="k">if</span> <span class="n">rows</span><span class="p">:</span>
|
||||
</span><span id="Table.__init__-17"><a href="#Table.__init__-17"><span class="linenos">17</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.__init__-18"><a href="#Table.__init__-18"><span class="linenos">18</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range_reader</span> <span class="o">=</span> <span class="n">RangeReader</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.__init__-12"><a href="#Table.__init__-12"><span class="linenos">12</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="Table.__init__-13"><a href="#Table.__init__-13"><span class="linenos">13</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.__init__-14"><a href="#Table.__init__-14"><span class="linenos">14</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="n">column_range</span>
|
||||
</span><span id="Table.__init__-15"><a href="#Table.__init__-15"><span class="linenos">15</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span><span id="Table.__init__-16"><a href="#Table.__init__-16"><span class="linenos">16</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span> <span class="o">=</span> <span class="n">rows</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="Table.__init__-17"><a href="#Table.__init__-17"><span class="linenos">17</span></a> <span class="k">if</span> <span class="n">rows</span><span class="p">:</span>
|
||||
</span><span id="Table.__init__-18"><a href="#Table.__init__-18"><span class="linenos">18</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rows</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.__init__-19"><a href="#Table.__init__-19"><span class="linenos">19</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range_reader</span> <span class="o">=</span> <span class="n">RangeReader</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -444,13 +456,13 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Table.add_columns"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.add_columns-20"><a href="#Table.add_columns-20"><span class="linenos">20</span></a> <span class="k">def</span> <span class="nf">add_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">columns</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Table.add_columns-21"><a href="#Table.add_columns-21"><span class="linenos">21</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">+=</span> <span class="n">columns</span>
|
||||
</span><span id="Table.add_columns-22"><a href="#Table.add_columns-22"><span class="linenos">22</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">:</span>
|
||||
</span><span id="Table.add_columns-23"><a href="#Table.add_columns-23"><span class="linenos">23</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span>
|
||||
</span><span id="Table.add_columns-24"><a href="#Table.add_columns-24"><span class="linenos">24</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">start</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">stop</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.add_columns-25"><a href="#Table.add_columns-25"><span class="linenos">25</span></a> <span class="p">)</span>
|
||||
</span><span id="Table.add_columns-26"><a href="#Table.add_columns-26"><span class="linenos">26</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.add_columns-21"><a href="#Table.add_columns-21"><span class="linenos">21</span></a> <span class="k">def</span> <span class="nf">add_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">columns</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Table.add_columns-22"><a href="#Table.add_columns-22"><span class="linenos">22</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">+=</span> <span class="n">columns</span>
|
||||
</span><span id="Table.add_columns-23"><a href="#Table.add_columns-23"><span class="linenos">23</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">:</span>
|
||||
</span><span id="Table.add_columns-24"><a href="#Table.add_columns-24"><span class="linenos">24</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span>
|
||||
</span><span id="Table.add_columns-25"><a href="#Table.add_columns-25"><span class="linenos">25</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">start</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="o">.</span><span class="n">stop</span> <span class="o">+</span> <span class="nb">len</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.add_columns-26"><a href="#Table.add_columns-26"><span class="linenos">26</span></a> <span class="p">)</span>
|
||||
</span><span id="Table.add_columns-27"><a href="#Table.add_columns-27"><span class="linenos">27</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span> <span class="o">=</span> <span class="n">RowReader</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -468,9 +480,9 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Table.append"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.append-28"><a href="#Table.append-28"><span class="linenos">28</span></a> <span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">row</span><span class="p">):</span>
|
||||
</span><span id="Table.append-29"><a href="#Table.append-29"><span class="linenos">29</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.append-30"><a href="#Table.append-30"><span class="linenos">30</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.append-29"><a href="#Table.append-29"><span class="linenos">29</span></a> <span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">row</span><span class="p">):</span>
|
||||
</span><span id="Table.append-30"><a href="#Table.append-30"><span class="linenos">30</span></a> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">row</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Table.append-31"><a href="#Table.append-31"><span class="linenos">31</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">row</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -488,8 +500,8 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Table.pop"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.pop-32"><a href="#Table.pop-32"><span class="linenos">32</span></a> <span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table.pop-33"><a href="#Table.pop-33"><span class="linenos">33</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.pop-33"><a href="#Table.pop-33"><span class="linenos">33</span></a> <span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Table.pop-34"><a href="#Table.pop-34"><span class="linenos">34</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -519,19 +531,19 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#TableIter"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter-70"><a href="#TableIter-70"><span class="linenos">70</span></a><span class="k">class</span> <span class="nc">TableIter</span><span class="p">:</span>
|
||||
</span><span id="TableIter-71"><a href="#TableIter-71"><span class="linenos">71</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="TableIter-72"><a href="#TableIter-72"><span class="linenos">72</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="TableIter-73"><a href="#TableIter-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
|
||||
</span><span id="TableIter-74"><a href="#TableIter-74"><span class="linenos">74</span></a>
|
||||
</span><span id="TableIter-75"><a href="#TableIter-75"><span class="linenos">75</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="TableIter-76"><a href="#TableIter-76"><span class="linenos">76</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||
</span><span id="TableIter-77"><a href="#TableIter-77"><span class="linenos">77</span></a>
|
||||
</span><span id="TableIter-78"><a href="#TableIter-78"><span class="linenos">78</span></a> <span class="k">def</span> <span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="TableIter-79"><a href="#TableIter-79"><span class="linenos">79</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
</span><span id="TableIter-80"><a href="#TableIter-80"><span class="linenos">80</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o"><</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="TableIter-81"><a href="#TableIter-81"><span class="linenos">81</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="TableIter-82"><a href="#TableIter-82"><span class="linenos">82</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter-71"><a href="#TableIter-71"><span class="linenos">71</span></a><span class="k">class</span> <span class="nc">TableIter</span><span class="p">:</span>
|
||||
</span><span id="TableIter-72"><a href="#TableIter-72"><span class="linenos">72</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="TableIter-73"><a href="#TableIter-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="TableIter-74"><a href="#TableIter-74"><span class="linenos">74</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
|
||||
</span><span id="TableIter-75"><a href="#TableIter-75"><span class="linenos">75</span></a>
|
||||
</span><span id="TableIter-76"><a href="#TableIter-76"><span class="linenos">76</span></a> <span class="k">def</span> <span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="TableIter-77"><a href="#TableIter-77"><span class="linenos">77</span></a> <span class="k">return</span> <span class="bp">self</span>
|
||||
</span><span id="TableIter-78"><a href="#TableIter-78"><span class="linenos">78</span></a>
|
||||
</span><span id="TableIter-79"><a href="#TableIter-79"><span class="linenos">79</span></a> <span class="k">def</span> <span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="TableIter-80"><a href="#TableIter-80"><span class="linenos">80</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
|
||||
</span><span id="TableIter-81"><a href="#TableIter-81"><span class="linenos">81</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o"><</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="TableIter-82"><a href="#TableIter-82"><span class="linenos">82</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
|
||||
</span><span id="TableIter-83"><a href="#TableIter-83"><span class="linenos">83</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -547,9 +559,9 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#TableIter.__init__"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter.__init__-71"><a href="#TableIter.__init__-71"><span class="linenos">71</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="TableIter.__init__-72"><a href="#TableIter.__init__-72"><span class="linenos">72</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="TableIter.__init__-73"><a href="#TableIter.__init__-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter.__init__-72"><a href="#TableIter.__init__-72"><span class="linenos">72</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="TableIter.__init__-73"><a href="#TableIter.__init__-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="TableIter.__init__-74"><a href="#TableIter.__init__-74"><span class="linenos">74</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -590,16 +602,16 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#RangeReader"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader-85"><a href="#RangeReader-85"><span class="linenos">85</span></a><span class="k">class</span> <span class="nc">RangeReader</span><span class="p">:</span>
|
||||
</span><span id="RangeReader-86"><a href="#RangeReader-86"><span class="linenos">86</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="RangeReader-87"><a href="#RangeReader-87"><span class="linenos">87</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="RangeReader-88"><a href="#RangeReader-88"><span class="linenos">88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="RangeReader-89"><a href="#RangeReader-89"><span class="linenos">89</span></a>
|
||||
</span><span id="RangeReader-90"><a href="#RangeReader-90"><span class="linenos">90</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="RangeReader-91"><a href="#RangeReader-91"><span class="linenos">91</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span><span id="RangeReader-92"><a href="#RangeReader-92"><span class="linenos">92</span></a>
|
||||
</span><span id="RangeReader-93"><a href="#RangeReader-93"><span class="linenos">93</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="RangeReader-94"><a href="#RangeReader-94"><span class="linenos">94</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader-86"><a href="#RangeReader-86"><span class="linenos">86</span></a><span class="k">class</span> <span class="nc">RangeReader</span><span class="p">:</span>
|
||||
</span><span id="RangeReader-87"><a href="#RangeReader-87"><span class="linenos">87</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="RangeReader-88"><a href="#RangeReader-88"><span class="linenos">88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="RangeReader-89"><a href="#RangeReader-89"><span class="linenos">89</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="RangeReader-90"><a href="#RangeReader-90"><span class="linenos">90</span></a>
|
||||
</span><span id="RangeReader-91"><a href="#RangeReader-91"><span class="linenos">91</span></a> <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="RangeReader-92"><a href="#RangeReader-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span><span id="RangeReader-93"><a href="#RangeReader-93"><span class="linenos">93</span></a>
|
||||
</span><span id="RangeReader-94"><a href="#RangeReader-94"><span class="linenos">94</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="RangeReader-95"><a href="#RangeReader-95"><span class="linenos">95</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -615,9 +627,9 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#RangeReader.__init__"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader.__init__-86"><a href="#RangeReader.__init__-86"><span class="linenos">86</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="RangeReader.__init__-87"><a href="#RangeReader.__init__-87"><span class="linenos">87</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="RangeReader.__init__-88"><a href="#RangeReader.__init__-88"><span class="linenos">88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader.__init__-87"><a href="#RangeReader.__init__-87"><span class="linenos">87</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
|
||||
</span><span id="RangeReader.__init__-88"><a href="#RangeReader.__init__-88"><span class="linenos">88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="RangeReader.__init__-89"><a href="#RangeReader.__init__-89"><span class="linenos">89</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -658,15 +670,15 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#RowReader"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader-97"><a href="#RowReader-97"><span class="linenos"> 97</span></a><span class="k">class</span> <span class="nc">RowReader</span><span class="p">:</span>
|
||||
</span><span id="RowReader-98"><a href="#RowReader-98"><span class="linenos"> 98</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="RowReader-99"><a href="#RowReader-99"><span class="linenos"> 99</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="RowReader-100"><a href="#RowReader-100"><span class="linenos">100</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
|
||||
</span><span id="RowReader-101"><a href="#RowReader-101"><span class="linenos">101</span></a> <span class="p">}</span>
|
||||
</span><span id="RowReader-102"><a href="#RowReader-102"><span class="linenos">102</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="RowReader-103"><a href="#RowReader-103"><span class="linenos">103</span></a>
|
||||
</span><span id="RowReader-104"><a href="#RowReader-104"><span class="linenos">104</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="RowReader-105"><a href="#RowReader-105"><span class="linenos">105</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader-98"><a href="#RowReader-98"><span class="linenos"> 98</span></a><span class="k">class</span> <span class="nc">RowReader</span><span class="p">:</span>
|
||||
</span><span id="RowReader-99"><a href="#RowReader-99"><span class="linenos"> 99</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="RowReader-100"><a href="#RowReader-100"><span class="linenos">100</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="RowReader-101"><a href="#RowReader-101"><span class="linenos">101</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
|
||||
</span><span id="RowReader-102"><a href="#RowReader-102"><span class="linenos">102</span></a> <span class="p">}</span>
|
||||
</span><span id="RowReader-103"><a href="#RowReader-103"><span class="linenos">103</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="RowReader-104"><a href="#RowReader-104"><span class="linenos">104</span></a>
|
||||
</span><span id="RowReader-105"><a href="#RowReader-105"><span class="linenos">105</span></a> <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="RowReader-106"><a href="#RowReader-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -682,11 +694,11 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#RowReader.__init__"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader.__init__-98"><a href="#RowReader.__init__-98"><span class="linenos"> 98</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="RowReader.__init__-99"><a href="#RowReader.__init__-99"><span class="linenos"> 99</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="RowReader.__init__-100"><a href="#RowReader.__init__-100"><span class="linenos">100</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
|
||||
</span><span id="RowReader.__init__-101"><a href="#RowReader.__init__-101"><span class="linenos">101</span></a> <span class="p">}</span>
|
||||
</span><span id="RowReader.__init__-102"><a href="#RowReader.__init__-102"><span class="linenos">102</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader.__init__-99"><a href="#RowReader.__init__-99"><span class="linenos"> 99</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||||
</span><span id="RowReader.__init__-100"><a href="#RowReader.__init__-100"><span class="linenos">100</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="RowReader.__init__-101"><a href="#RowReader.__init__-101"><span class="linenos">101</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
|
||||
</span><span id="RowReader.__init__-102"><a href="#RowReader.__init__-102"><span class="linenos">102</span></a> <span class="p">}</span>
|
||||
</span><span id="RowReader.__init__-103"><a href="#RowReader.__init__-103"><span class="linenos">103</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -727,8 +739,8 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Tables"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Tables-108"><a href="#Tables-108"><span class="linenos">108</span></a><span class="k">class</span> <span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">[</span><span class="n">Table</span><span class="p">]):</span>
|
||||
</span><span id="Tables-109"><a href="#Tables-109"><span class="linenos">109</span></a> <span class="k">pass</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Tables-109"><a href="#Tables-109"><span class="linenos">109</span></a><span class="k">class</span> <span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">[</span><span class="n">Table</span><span class="p">]):</span>
|
||||
</span><span id="Tables-110"><a href="#Tables-110"><span class="linenos">110</span></a> <span class="k">pass</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -776,14 +788,14 @@ For example, a generic mapping type might be defined as::</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">ensure_tables</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">d</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Dict</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Tables">sqlglot.executor.table.Tables</a></span>:</span></span>
|
||||
<span class="name">ensure_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">d</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#Tables">sqlglot.executor.table.Tables</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="ensure_tables-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#ensure_tables"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_tables-112"><a href="#ensure_tables-112"><span class="linenos">112</span></a><span class="k">def</span> <span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">])</span> <span class="o">-></span> <span class="n">Tables</span><span class="p">:</span>
|
||||
</span><span id="ensure_tables-113"><a href="#ensure_tables-113"><span class="linenos">113</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">))</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_tables-113"><a href="#ensure_tables-113"><span class="linenos">113</span></a><span class="k">def</span> <span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tables</span><span class="p">:</span>
|
||||
</span><span id="ensure_tables-114"><a href="#ensure_tables-114"><span class="linenos">114</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
|
|
@ -72,7 +72,14 @@
|
|||
<label class="view-source-button" for="mod-optimizer-view-source"><span>View Source</span></label>
|
||||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos">1</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.optimizer</span> <span class="kn">import</span> <span class="n">RULES</span><span class="p">,</span> <span class="n">optimize</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">build_scope</span><span class="p">,</span> <span class="n">traverse_scope</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="p">(</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a> <span class="n">Scope</span><span class="p">,</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a> <span class="n">build_scope</span><span class="p">,</span>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos">5</span></a> <span class="n">find_all_in_scope</span><span class="p">,</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos">6</span></a> <span class="n">find_in_scope</span><span class="p">,</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos">7</span></a> <span class="n">traverse_scope</span><span class="p">,</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos">8</span></a> <span class="n">walk_in_scope</span><span class="p">,</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos">9</span></a><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -200,65 +200,66 @@
|
|||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">pivots</span> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">):</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">parent</span><span class="o">.</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">parent</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">))</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="c1"># Get rid of redundant exp.Subquery expressions, i.e. those that are just used as wrappers</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">unwrap</span><span class="p">()</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">to_replace</span><span class="o">.</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">to_replace</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">))</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a><span class="k">def</span> <span class="nf">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="c1"># Rename references to this CTE</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">traverse</span><span class="p">():</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">child_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="n">source</span> <span class="ow">is</span> <span class="n">scope</span><span class="p">:</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">new_table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">table</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_table</span><span class="p">)</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="k">def</span> <span class="nf">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="c1"># Rename references to this CTE</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">traverse</span><span class="p">():</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">child_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">if</span> <span class="n">source</span> <span class="ow">is</span> <span class="n">scope</span><span class="p">:</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">new_table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">table</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_table</span><span class="p">)</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a><span class="k">def</span> <span class="nf">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="sd"> tuple of (name, cte)</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a><span class="sd"> where `name` is a new name for this CTE in the root scope and `cte` is a new CTE instance.</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a><span class="sd"> If this CTE duplicates an existing CTE, `cte` will be None.</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">duplicate_cte_alias</span> <span class="o">=</span> <span class="n">existing_ctes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s2">"cte"</span><span class="p">)</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="k">if</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">duplicate_cte_alias</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="k">elif</span> <span class="n">taken</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">existing_ctes</span><span class="p">[</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">]</span> <span class="o">=</span> <span class="n">name</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">(</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="n">this</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="p">)</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="k">return</span> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a><span class="k">def</span> <span class="nf">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a><span class="sd"> tuple of (name, cte)</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a><span class="sd"> where `name` is a new name for this CTE in the root scope and `cte` is a new CTE instance.</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a><span class="sd"> If this CTE duplicates an existing CTE, `cte` will be None.</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">duplicate_cte_alias</span> <span class="o">=</span> <span class="n">existing_ctes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s2">"cte"</span><span class="p">)</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="k">if</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">duplicate_cte_alias</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="k">elif</span> <span class="n">taken</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="n">existing_ctes</span><span class="p">[</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">]</span> <span class="o">=</span> <span class="n">name</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">(</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="n">this</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="p">)</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="k">return</span> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -568,7 +568,7 @@ queries if it would result in multiple table selects in a single query:</p>
|
|||
<div class="attr variable">
|
||||
<span class="name">UNMERGABLE_ARGS</span> =
|
||||
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'sample', 'distribute', 'group', 'into', 'kind', 'settings', 'format', 'offset', 'windows', 'pivots', 'qualify', 'having', 'distinct', 'with', 'limit', 'locks', 'laterals', 'cluster', 'sort', 'match'}</span>
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'kind', 'format', 'having', 'locks', 'distinct', 'distribute', 'with', 'offset', 'settings', 'sample', 'cluster', 'sort', 'group', 'into', 'connect', 'pivots', 'match', 'limit', 'windows', 'qualify', 'laterals'}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -142,14 +142,19 @@
|
|||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">JOIN_ATTRS</span><span class="p">):</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"CROSS"</span><span class="p">)</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">!=</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">==</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">if</span> <span class="n">on</span> <span class="k">else</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"using"</span><span class="p">):</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">if</span> <span class="n">on</span> <span class="k">else</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -286,9 +291,14 @@
|
|||
</span><span id="normalize-73"><a href="#normalize-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">JOIN_ATTRS</span><span class="p">):</span>
|
||||
</span><span id="normalize-74"><a href="#normalize-74"><span class="linenos">74</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"CROSS"</span><span class="p">)</span>
|
||||
</span><span id="normalize-75"><a href="#normalize-75"><span class="linenos">75</span></a>
|
||||
</span><span id="normalize-76"><a href="#normalize-76"><span class="linenos">76</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">!=</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="normalize-77"><a href="#normalize-77"><span class="linenos">77</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="normalize-78"><a href="#normalize-78"><span class="linenos">78</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="normalize-76"><a href="#normalize-76"><span class="linenos">76</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">==</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="normalize-77"><a href="#normalize-77"><span class="linenos">77</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="normalize-78"><a href="#normalize-78"><span class="linenos">78</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="normalize-79"><a href="#normalize-79"><span class="linenos">79</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="normalize-80"><a href="#normalize-80"><span class="linenos">80</span></a>
|
||||
</span><span id="normalize-81"><a href="#normalize-81"><span class="linenos">81</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"using"</span><span class="p">):</span>
|
||||
</span><span id="normalize-82"><a href="#normalize-82"><span class="linenos">82</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="normalize-83"><a href="#normalize-83"><span class="linenos">83</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -308,9 +318,9 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#other_table_names"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="other_table_names-81"><a href="#other_table_names-81"><span class="linenos">81</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="other_table_names-82"><a href="#other_table_names-82"><span class="linenos">82</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="other_table_names-83"><a href="#other_table_names-83"><span class="linenos">83</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">if</span> <span class="n">on</span> <span class="k">else</span> <span class="nb">set</span><span class="p">()</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="other_table_names-86"><a href="#other_table_names-86"><span class="linenos">86</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="other_table_names-87"><a href="#other_table_names-87"><span class="linenos">87</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="other_table_names-88"><a href="#other_table_names-88"><span class="linenos">88</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">if</span> <span class="n">on</span> <span class="k">else</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -73,7 +73,7 @@
|
|||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.normalize</span> <span class="kn">import</span> <span class="n">normalized</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">build_scope</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">build_scope</span><span class="p">,</span> <span class="n">find_in_scope</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.simplify</span> <span class="kn">import</span> <span class="n">simplify</span>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a>
|
||||
|
|
@ -154,125 +154,133 @@
|
|||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">break</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">)</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="p">)</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="p">)</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="p">)</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="p">):</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="p">)</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="p">):</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -422,7 +430,11 @@
|
|||
</span><span id="pushdown_cnf-82"><a href="#pushdown_cnf-82"><span class="linenos">82</span></a> <span class="k">break</span>
|
||||
</span><span id="pushdown_cnf-83"><a href="#pushdown_cnf-83"><span class="linenos">83</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-84"><a href="#pushdown_cnf-84"><span class="linenos">84</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="pushdown_cnf-85"><a href="#pushdown_cnf-85"><span class="linenos">85</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_cnf-85"><a href="#pushdown_cnf-85"><span class="linenos">85</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="pushdown_cnf-86"><a href="#pushdown_cnf-86"><span class="linenos">86</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-87"><a href="#pushdown_cnf-87"><span class="linenos">87</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_cnf-88"><a href="#pushdown_cnf-88"><span class="linenos">88</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="pushdown_cnf-89"><a href="#pushdown_cnf-89"><span class="linenos">89</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -442,65 +454,69 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#pushdown_dnf"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_dnf-88"><a href="#pushdown_dnf-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-89"><a href="#pushdown_dnf-89"><span class="linenos"> 89</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pushdown_dnf-90"><a href="#pushdown_dnf-90"><span class="linenos"> 90</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="pushdown_dnf-91"><a href="#pushdown_dnf-91"><span class="linenos"> 91</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="pushdown_dnf-92"><a href="#pushdown_dnf-92"><span class="linenos"> 92</span></a><span class="sd"> """</span>
|
||||
</span><span id="pushdown_dnf-93"><a href="#pushdown_dnf-93"><span class="linenos"> 93</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="pushdown_dnf-94"><a href="#pushdown_dnf-94"><span class="linenos"> 94</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="pushdown_dnf-95"><a href="#pushdown_dnf-95"><span class="linenos"> 95</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="pushdown_dnf-96"><a href="#pushdown_dnf-96"><span class="linenos"> 96</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="pushdown_dnf-97"><a href="#pushdown_dnf-97"><span class="linenos"> 97</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_dnf-98"><a href="#pushdown_dnf-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="pushdown_dnf-99"><a href="#pushdown_dnf-99"><span class="linenos"> 99</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-100"><a href="#pushdown_dnf-100"><span class="linenos">100</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-101"><a href="#pushdown_dnf-101"><span class="linenos">101</span></a>
|
||||
</span><span id="pushdown_dnf-102"><a href="#pushdown_dnf-102"><span class="linenos">102</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-103"><a href="#pushdown_dnf-103"><span class="linenos">103</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-104"><a href="#pushdown_dnf-104"><span class="linenos">104</span></a>
|
||||
</span><span id="pushdown_dnf-105"><a href="#pushdown_dnf-105"><span class="linenos">105</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-106"><a href="#pushdown_dnf-106"><span class="linenos">106</span></a>
|
||||
</span><span id="pushdown_dnf-107"><a href="#pushdown_dnf-107"><span class="linenos">107</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_dnf-92"><a href="#pushdown_dnf-92"><span class="linenos"> 92</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-93"><a href="#pushdown_dnf-93"><span class="linenos"> 93</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pushdown_dnf-94"><a href="#pushdown_dnf-94"><span class="linenos"> 94</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="pushdown_dnf-95"><a href="#pushdown_dnf-95"><span class="linenos"> 95</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="pushdown_dnf-96"><a href="#pushdown_dnf-96"><span class="linenos"> 96</span></a><span class="sd"> """</span>
|
||||
</span><span id="pushdown_dnf-97"><a href="#pushdown_dnf-97"><span class="linenos"> 97</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="pushdown_dnf-98"><a href="#pushdown_dnf-98"><span class="linenos"> 98</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="pushdown_dnf-99"><a href="#pushdown_dnf-99"><span class="linenos"> 99</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="pushdown_dnf-100"><a href="#pushdown_dnf-100"><span class="linenos">100</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="pushdown_dnf-101"><a href="#pushdown_dnf-101"><span class="linenos">101</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_dnf-102"><a href="#pushdown_dnf-102"><span class="linenos">102</span></a>
|
||||
</span><span id="pushdown_dnf-103"><a href="#pushdown_dnf-103"><span class="linenos">103</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-104"><a href="#pushdown_dnf-104"><span class="linenos">104</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-105"><a href="#pushdown_dnf-105"><span class="linenos">105</span></a>
|
||||
</span><span id="pushdown_dnf-106"><a href="#pushdown_dnf-106"><span class="linenos">106</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-107"><a href="#pushdown_dnf-107"><span class="linenos">107</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-108"><a href="#pushdown_dnf-108"><span class="linenos">108</span></a>
|
||||
</span><span id="pushdown_dnf-109"><a href="#pushdown_dnf-109"><span class="linenos">109</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="pushdown_dnf-110"><a href="#pushdown_dnf-110"><span class="linenos">110</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="pushdown_dnf-111"><a href="#pushdown_dnf-111"><span class="linenos">111</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="pushdown_dnf-112"><a href="#pushdown_dnf-112"><span class="linenos">112</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-113"><a href="#pushdown_dnf-113"><span class="linenos">113</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-114"><a href="#pushdown_dnf-114"><span class="linenos">114</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-115"><a href="#pushdown_dnf-115"><span class="linenos">115</span></a>
|
||||
</span><span id="pushdown_dnf-116"><a href="#pushdown_dnf-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-117"><a href="#pushdown_dnf-117"><span class="linenos">117</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-118"><a href="#pushdown_dnf-118"><span class="linenos">118</span></a>
|
||||
</span><span id="pushdown_dnf-119"><a href="#pushdown_dnf-119"><span class="linenos">119</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="pushdown_dnf-120"><a href="#pushdown_dnf-120"><span class="linenos">120</span></a>
|
||||
</span><span id="pushdown_dnf-121"><a href="#pushdown_dnf-121"><span class="linenos">121</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-122"><a href="#pushdown_dnf-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-123"><a href="#pushdown_dnf-123"><span class="linenos">123</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-124"><a href="#pushdown_dnf-124"><span class="linenos">124</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-125"><a href="#pushdown_dnf-125"><span class="linenos">125</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-126"><a href="#pushdown_dnf-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-127"><a href="#pushdown_dnf-127"><span class="linenos">127</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="pushdown_dnf-128"><a href="#pushdown_dnf-128"><span class="linenos">128</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-129"><a href="#pushdown_dnf-129"><span class="linenos">129</span></a>
|
||||
</span><span id="pushdown_dnf-130"><a href="#pushdown_dnf-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-131"><a href="#pushdown_dnf-131"><span class="linenos">131</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-132"><a href="#pushdown_dnf-132"><span class="linenos">132</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-133"><a href="#pushdown_dnf-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="pushdown_dnf-134"><a href="#pushdown_dnf-134"><span class="linenos">134</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-135"><a href="#pushdown_dnf-135"><span class="linenos">135</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-136"><a href="#pushdown_dnf-136"><span class="linenos">136</span></a>
|
||||
</span><span id="pushdown_dnf-137"><a href="#pushdown_dnf-137"><span class="linenos">137</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="pushdown_dnf-138"><a href="#pushdown_dnf-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-139"><a href="#pushdown_dnf-139"><span class="linenos">139</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-109"><a href="#pushdown_dnf-109"><span class="linenos">109</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-110"><a href="#pushdown_dnf-110"><span class="linenos">110</span></a>
|
||||
</span><span id="pushdown_dnf-111"><a href="#pushdown_dnf-111"><span class="linenos">111</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="pushdown_dnf-112"><a href="#pushdown_dnf-112"><span class="linenos">112</span></a>
|
||||
</span><span id="pushdown_dnf-113"><a href="#pushdown_dnf-113"><span class="linenos">113</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="pushdown_dnf-114"><a href="#pushdown_dnf-114"><span class="linenos">114</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="pushdown_dnf-115"><a href="#pushdown_dnf-115"><span class="linenos">115</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="pushdown_dnf-116"><a href="#pushdown_dnf-116"><span class="linenos">116</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-117"><a href="#pushdown_dnf-117"><span class="linenos">117</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-118"><a href="#pushdown_dnf-118"><span class="linenos">118</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-119"><a href="#pushdown_dnf-119"><span class="linenos">119</span></a>
|
||||
</span><span id="pushdown_dnf-120"><a href="#pushdown_dnf-120"><span class="linenos">120</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-121"><a href="#pushdown_dnf-121"><span class="linenos">121</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-122"><a href="#pushdown_dnf-122"><span class="linenos">122</span></a>
|
||||
</span><span id="pushdown_dnf-123"><a href="#pushdown_dnf-123"><span class="linenos">123</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="pushdown_dnf-124"><a href="#pushdown_dnf-124"><span class="linenos">124</span></a>
|
||||
</span><span id="pushdown_dnf-125"><a href="#pushdown_dnf-125"><span class="linenos">125</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-126"><a href="#pushdown_dnf-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-127"><a href="#pushdown_dnf-127"><span class="linenos">127</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-128"><a href="#pushdown_dnf-128"><span class="linenos">128</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-129"><a href="#pushdown_dnf-129"><span class="linenos">129</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-130"><a href="#pushdown_dnf-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-131"><a href="#pushdown_dnf-131"><span class="linenos">131</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="pushdown_dnf-132"><a href="#pushdown_dnf-132"><span class="linenos">132</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-133"><a href="#pushdown_dnf-133"><span class="linenos">133</span></a>
|
||||
</span><span id="pushdown_dnf-134"><a href="#pushdown_dnf-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-135"><a href="#pushdown_dnf-135"><span class="linenos">135</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-136"><a href="#pushdown_dnf-136"><span class="linenos">136</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-137"><a href="#pushdown_dnf-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="pushdown_dnf-138"><a href="#pushdown_dnf-138"><span class="linenos">138</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-139"><a href="#pushdown_dnf-139"><span class="linenos">139</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-140"><a href="#pushdown_dnf-140"><span class="linenos">140</span></a>
|
||||
</span><span id="pushdown_dnf-141"><a href="#pushdown_dnf-141"><span class="linenos">141</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="pushdown_dnf-142"><a href="#pushdown_dnf-142"><span class="linenos">142</span></a>
|
||||
</span><span id="pushdown_dnf-143"><a href="#pushdown_dnf-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-144"><a href="#pushdown_dnf-144"><span class="linenos">144</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-145"><a href="#pushdown_dnf-145"><span class="linenos">145</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-146"><a href="#pushdown_dnf-146"><span class="linenos">146</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-141"><a href="#pushdown_dnf-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="pushdown_dnf-142"><a href="#pushdown_dnf-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-143"><a href="#pushdown_dnf-143"><span class="linenos">143</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-144"><a href="#pushdown_dnf-144"><span class="linenos">144</span></a>
|
||||
</span><span id="pushdown_dnf-145"><a href="#pushdown_dnf-145"><span class="linenos">145</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="pushdown_dnf-146"><a href="#pushdown_dnf-146"><span class="linenos">146</span></a>
|
||||
</span><span id="pushdown_dnf-147"><a href="#pushdown_dnf-147"><span class="linenos">147</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-148"><a href="#pushdown_dnf-148"><span class="linenos">148</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-149"><a href="#pushdown_dnf-149"><span class="linenos">149</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-150"><a href="#pushdown_dnf-150"><span class="linenos">150</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-151"><a href="#pushdown_dnf-151"><span class="linenos">151</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-152"><a href="#pushdown_dnf-152"><span class="linenos">152</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-153"><a href="#pushdown_dnf-153"><span class="linenos">153</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-154"><a href="#pushdown_dnf-154"><span class="linenos">154</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -521,44 +537,44 @@ Additionally, we can't remove predicates from their original form.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#nodes_for_predicate"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nodes_for_predicate-149"><a href="#nodes_for_predicate-149"><span class="linenos">149</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-150"><a href="#nodes_for_predicate-150"><span class="linenos">150</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-151"><a href="#nodes_for_predicate-151"><span class="linenos">151</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-152"><a href="#nodes_for_predicate-152"><span class="linenos">152</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-153"><a href="#nodes_for_predicate-153"><span class="linenos">153</span></a>
|
||||
</span><span id="nodes_for_predicate-154"><a href="#nodes_for_predicate-154"><span class="linenos">154</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-155"><a href="#nodes_for_predicate-155"><span class="linenos">155</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-156"><a href="#nodes_for_predicate-156"><span class="linenos">156</span></a>
|
||||
</span><span id="nodes_for_predicate-157"><a href="#nodes_for_predicate-157"><span class="linenos">157</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="nodes_for_predicate-158"><a href="#nodes_for_predicate-158"><span class="linenos">158</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="nodes_for_predicate-159"><a href="#nodes_for_predicate-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-160"><a href="#nodes_for_predicate-160"><span class="linenos">160</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nodes_for_predicate-157"><a href="#nodes_for_predicate-157"><span class="linenos">157</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-158"><a href="#nodes_for_predicate-158"><span class="linenos">158</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-159"><a href="#nodes_for_predicate-159"><span class="linenos">159</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-160"><a href="#nodes_for_predicate-160"><span class="linenos">160</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-161"><a href="#nodes_for_predicate-161"><span class="linenos">161</span></a>
|
||||
</span><span id="nodes_for_predicate-162"><a href="#nodes_for_predicate-162"><span class="linenos">162</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="nodes_for_predicate-163"><a href="#nodes_for_predicate-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-164"><a href="#nodes_for_predicate-164"><span class="linenos">164</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-165"><a href="#nodes_for_predicate-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-166"><a href="#nodes_for_predicate-166"><span class="linenos">166</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-167"><a href="#nodes_for_predicate-167"><span class="linenos">167</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="nodes_for_predicate-168"><a href="#nodes_for_predicate-168"><span class="linenos">168</span></a>
|
||||
</span><span id="nodes_for_predicate-169"><a href="#nodes_for_predicate-169"><span class="linenos">169</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-170"><a href="#nodes_for_predicate-170"><span class="linenos">170</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-171"><a href="#nodes_for_predicate-171"><span class="linenos">171</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-172"><a href="#nodes_for_predicate-172"><span class="linenos">172</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-173"><a href="#nodes_for_predicate-173"><span class="linenos">173</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-174"><a href="#nodes_for_predicate-174"><span class="linenos">174</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="nodes_for_predicate-175"><a href="#nodes_for_predicate-175"><span class="linenos">175</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-176"><a href="#nodes_for_predicate-176"><span class="linenos">176</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-177"><a href="#nodes_for_predicate-177"><span class="linenos">177</span></a> <span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-178"><a href="#nodes_for_predicate-178"><span class="linenos">178</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="nodes_for_predicate-179"><a href="#nodes_for_predicate-179"><span class="linenos">179</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="nodes_for_predicate-180"><a href="#nodes_for_predicate-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-181"><a href="#nodes_for_predicate-181"><span class="linenos">181</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-182"><a href="#nodes_for_predicate-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="nodes_for_predicate-183"><a href="#nodes_for_predicate-183"><span class="linenos">183</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="nodes_for_predicate-184"><a href="#nodes_for_predicate-184"><span class="linenos">184</span></a> <span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-185"><a href="#nodes_for_predicate-185"><span class="linenos">185</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-186"><a href="#nodes_for_predicate-186"><span class="linenos">186</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span><span id="nodes_for_predicate-162"><a href="#nodes_for_predicate-162"><span class="linenos">162</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-163"><a href="#nodes_for_predicate-163"><span class="linenos">163</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-164"><a href="#nodes_for_predicate-164"><span class="linenos">164</span></a>
|
||||
</span><span id="nodes_for_predicate-165"><a href="#nodes_for_predicate-165"><span class="linenos">165</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="nodes_for_predicate-166"><a href="#nodes_for_predicate-166"><span class="linenos">166</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="nodes_for_predicate-167"><a href="#nodes_for_predicate-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-168"><a href="#nodes_for_predicate-168"><span class="linenos">168</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-169"><a href="#nodes_for_predicate-169"><span class="linenos">169</span></a>
|
||||
</span><span id="nodes_for_predicate-170"><a href="#nodes_for_predicate-170"><span class="linenos">170</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="nodes_for_predicate-171"><a href="#nodes_for_predicate-171"><span class="linenos">171</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-172"><a href="#nodes_for_predicate-172"><span class="linenos">172</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-173"><a href="#nodes_for_predicate-173"><span class="linenos">173</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-174"><a href="#nodes_for_predicate-174"><span class="linenos">174</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-175"><a href="#nodes_for_predicate-175"><span class="linenos">175</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="nodes_for_predicate-176"><a href="#nodes_for_predicate-176"><span class="linenos">176</span></a>
|
||||
</span><span id="nodes_for_predicate-177"><a href="#nodes_for_predicate-177"><span class="linenos">177</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-178"><a href="#nodes_for_predicate-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-179"><a href="#nodes_for_predicate-179"><span class="linenos">179</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-180"><a href="#nodes_for_predicate-180"><span class="linenos">180</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-181"><a href="#nodes_for_predicate-181"><span class="linenos">181</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-182"><a href="#nodes_for_predicate-182"><span class="linenos">182</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="nodes_for_predicate-183"><a href="#nodes_for_predicate-183"><span class="linenos">183</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-184"><a href="#nodes_for_predicate-184"><span class="linenos">184</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-185"><a href="#nodes_for_predicate-185"><span class="linenos">185</span></a> <span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-186"><a href="#nodes_for_predicate-186"><span class="linenos">186</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="nodes_for_predicate-187"><a href="#nodes_for_predicate-187"><span class="linenos">187</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="nodes_for_predicate-188"><a href="#nodes_for_predicate-188"><span class="linenos">188</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-189"><a href="#nodes_for_predicate-189"><span class="linenos">189</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-190"><a href="#nodes_for_predicate-190"><span class="linenos">190</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="nodes_for_predicate-191"><a href="#nodes_for_predicate-191"><span class="linenos">191</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="nodes_for_predicate-192"><a href="#nodes_for_predicate-192"><span class="linenos">192</span></a> <span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-193"><a href="#nodes_for_predicate-193"><span class="linenos">193</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-194"><a href="#nodes_for_predicate-194"><span class="linenos">194</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -576,21 +592,21 @@ Additionally, we can't remove predicates from their original form.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#replace_aliases"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_aliases-189"><a href="#replace_aliases-189"><span class="linenos">189</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-190"><a href="#replace_aliases-190"><span class="linenos">190</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="replace_aliases-191"><a href="#replace_aliases-191"><span class="linenos">191</span></a>
|
||||
</span><span id="replace_aliases-192"><a href="#replace_aliases-192"><span class="linenos">192</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-193"><a href="#replace_aliases-193"><span class="linenos">193</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-194"><a href="#replace_aliases-194"><span class="linenos">194</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="replace_aliases-195"><a href="#replace_aliases-195"><span class="linenos">195</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-196"><a href="#replace_aliases-196"><span class="linenos">196</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="replace_aliases-197"><a href="#replace_aliases-197"><span class="linenos">197</span></a>
|
||||
</span><span id="replace_aliases-198"><a href="#replace_aliases-198"><span class="linenos">198</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-199"><a href="#replace_aliases-199"><span class="linenos">199</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-200"><a href="#replace_aliases-200"><span class="linenos">200</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="replace_aliases-201"><a href="#replace_aliases-201"><span class="linenos">201</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="replace_aliases-202"><a href="#replace_aliases-202"><span class="linenos">202</span></a>
|
||||
</span><span id="replace_aliases-203"><a href="#replace_aliases-203"><span class="linenos">203</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_aliases-197"><a href="#replace_aliases-197"><span class="linenos">197</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-198"><a href="#replace_aliases-198"><span class="linenos">198</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="replace_aliases-199"><a href="#replace_aliases-199"><span class="linenos">199</span></a>
|
||||
</span><span id="replace_aliases-200"><a href="#replace_aliases-200"><span class="linenos">200</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-201"><a href="#replace_aliases-201"><span class="linenos">201</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-202"><a href="#replace_aliases-202"><span class="linenos">202</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="replace_aliases-203"><a href="#replace_aliases-203"><span class="linenos">203</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-204"><a href="#replace_aliases-204"><span class="linenos">204</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="replace_aliases-205"><a href="#replace_aliases-205"><span class="linenos">205</span></a>
|
||||
</span><span id="replace_aliases-206"><a href="#replace_aliases-206"><span class="linenos">206</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-207"><a href="#replace_aliases-207"><span class="linenos">207</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-208"><a href="#replace_aliases-208"><span class="linenos">208</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="replace_aliases-209"><a href="#replace_aliases-209"><span class="linenos">209</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="replace_aliases-210"><a href="#replace_aliases-210"><span class="linenos">210</span></a>
|
||||
</span><span id="replace_aliases-211"><a href="#replace_aliases-211"><span class="linenos">211</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
19314
docs/sqlglot/parser.html
19314
docs/sqlglot/parser.html
File diff suppressed because one or more lines are too long
|
|
@ -114,6 +114,9 @@
|
|||
</ul>
|
||||
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#normalize_name">normalize_name</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#ensure_schema">ensure_schema</a>
|
||||
</li>
|
||||
|
|
@ -526,155 +529,166 @@
|
|||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="n">normalize</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a> <span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="n">normalize</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize</span> <span class="k">if</span> <span class="n">normalize</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">normalize</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">normalize</span><span class="p">:</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a> <span class="c1"># This can be useful for normalize_identifier</span>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a> <span class="n">identifier</span><span class="o">.</span><span class="n">meta</span><span class="p">[</span><span class="s2">"is_table"</span><span class="p">]</span> <span class="o">=</span> <span class="n">is_table</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a> <span class="k">def</span> <span class="nf">depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">empty</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span><span class="p">:</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a>
|
||||
</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a>
|
||||
</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="L-407"><a href="#L-407"><span class="linenos">407</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="k">return</span> <span class="n">normalize_name</span><span class="p">(</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="n">name</span><span class="p">,</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">,</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="n">is_table</span><span class="o">=</span><span class="n">is_table</span><span class="p">,</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">normalize</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">normalize</span> <span class="k">if</span> <span class="n">normalize</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">normalize</span><span class="p">,</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="p">)</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="k">def</span> <span class="nf">depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">empty</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span><span class="p">:</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a>
|
||||
</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="L-407"><a href="#L-407"><span class="linenos">407</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a>
|
||||
</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span><span id="L-410"><a href="#L-410"><span class="linenos">410</span></a>
|
||||
</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-413"><a href="#L-413"><span class="linenos">413</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a>
|
||||
</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a>
|
||||
</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a>
|
||||
</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a>
|
||||
</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a>
|
||||
</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a><span class="k">def</span> <span class="nf">normalize_name</span><span class="p">(</span>
|
||||
</span><span id="L-413"><a href="#L-413"><span class="linenos">413</span></a> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">,</span>
|
||||
</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a> <span class="n">normalize</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a>
|
||||
</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">normalize</span><span class="p">:</span>
|
||||
</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a>
|
||||
</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a>
|
||||
</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a> <span class="p">}</span>
|
||||
</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a>
|
||||
</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a>
|
||||
</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a>
|
||||
</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a>
|
||||
</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a>
|
||||
</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="L-464"><a href="#L-464"><span class="linenos">464</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a>
|
||||
</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a>
|
||||
</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="L-477"><a href="#L-477"><span class="linenos">477</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-478"><a href="#L-478"><span class="linenos">478</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="L-479"><a href="#L-479"><span class="linenos">479</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-480"><a href="#L-480"><span class="linenos">480</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-481"><a href="#L-481"><span class="linenos">481</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="L-482"><a href="#L-482"><span class="linenos">482</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="L-483"><a href="#L-483"><span class="linenos">483</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-484"><a href="#L-484"><span class="linenos">484</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a> <span class="c1"># This can be useful for normalize_identifier</span>
|
||||
</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a> <span class="n">identifier</span><span class="o">.</span><span class="n">meta</span><span class="p">[</span><span class="s2">"is_table"</span><span class="p">]</span> <span class="o">=</span> <span class="n">is_table</span>
|
||||
</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a>
|
||||
</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a>
|
||||
</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a>
|
||||
</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a>
|
||||
</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a>
|
||||
</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="p">}</span>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a>
|
||||
</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a>
|
||||
</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a>
|
||||
</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-464"><a href="#L-464"><span class="linenos">464</span></a>
|
||||
</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a>
|
||||
</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a>
|
||||
</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a>
|
||||
</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="L-477"><a href="#L-477"><span class="linenos">477</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-478"><a href="#L-478"><span class="linenos">478</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="L-479"><a href="#L-479"><span class="linenos">479</span></a>
|
||||
</span><span id="L-480"><a href="#L-480"><span class="linenos">480</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-481"><a href="#L-481"><span class="linenos">481</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="L-482"><a href="#L-482"><span class="linenos">482</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="L-483"><a href="#L-483"><span class="linenos">483</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="L-484"><a href="#L-484"><span class="linenos">484</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="L-485"><a href="#L-485"><span class="linenos">485</span></a>
|
||||
</span><span id="L-486"><a href="#L-486"><span class="linenos">486</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-487"><a href="#L-487"><span class="linenos">487</span></a>
|
||||
</span><span id="L-488"><a href="#L-488"><span class="linenos">488</span></a>
|
||||
</span><span id="L-489"><a href="#L-489"><span class="linenos">489</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-490"><a href="#L-490"><span class="linenos">490</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-491"><a href="#L-491"><span class="linenos">491</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="L-492"><a href="#L-492"><span class="linenos">492</span></a>
|
||||
</span><span id="L-493"><a href="#L-493"><span class="linenos">493</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-494"><a href="#L-494"><span class="linenos">494</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-495"><a href="#L-495"><span class="linenos">495</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="L-486"><a href="#L-486"><span class="linenos">486</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-487"><a href="#L-487"><span class="linenos">487</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="L-488"><a href="#L-488"><span class="linenos">488</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-489"><a href="#L-489"><span class="linenos">489</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="L-490"><a href="#L-490"><span class="linenos">490</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-491"><a href="#L-491"><span class="linenos">491</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-492"><a href="#L-492"><span class="linenos">492</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="L-493"><a href="#L-493"><span class="linenos">493</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="L-494"><a href="#L-494"><span class="linenos">494</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-495"><a href="#L-495"><span class="linenos">495</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="L-496"><a href="#L-496"><span class="linenos">496</span></a>
|
||||
</span><span id="L-497"><a href="#L-497"><span class="linenos">497</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-498"><a href="#L-498"><span class="linenos">498</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="L-497"><a href="#L-497"><span class="linenos">497</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-498"><a href="#L-498"><span class="linenos">498</span></a>
|
||||
</span><span id="L-499"><a href="#L-499"><span class="linenos">499</span></a>
|
||||
</span><span id="L-500"><a href="#L-500"><span class="linenos">500</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-501"><a href="#L-501"><span class="linenos">501</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="L-502"><a href="#L-502"><span class="linenos">502</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="L-503"><a href="#L-503"><span class="linenos">503</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="L-504"><a href="#L-504"><span class="linenos">504</span></a>
|
||||
</span><span id="L-505"><a href="#L-505"><span class="linenos">505</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-506"><a href="#L-506"><span class="linenos">506</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="L-507"><a href="#L-507"><span class="linenos">507</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-508"><a href="#L-508"><span class="linenos">508</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="L-509"><a href="#L-509"><span class="linenos">509</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-500"><a href="#L-500"><span class="linenos">500</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-501"><a href="#L-501"><span class="linenos">501</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-502"><a href="#L-502"><span class="linenos">502</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="L-503"><a href="#L-503"><span class="linenos">503</span></a>
|
||||
</span><span id="L-504"><a href="#L-504"><span class="linenos">504</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-505"><a href="#L-505"><span class="linenos">505</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-506"><a href="#L-506"><span class="linenos">506</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="L-507"><a href="#L-507"><span class="linenos">507</span></a>
|
||||
</span><span id="L-508"><a href="#L-508"><span class="linenos">508</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-509"><a href="#L-509"><span class="linenos">509</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="L-510"><a href="#L-510"><span class="linenos">510</span></a>
|
||||
</span><span id="L-511"><a href="#L-511"><span class="linenos">511</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-512"><a href="#L-512"><span class="linenos">512</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-513"><a href="#L-513"><span class="linenos">513</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-514"><a href="#L-514"><span class="linenos">514</span></a>
|
||||
</span><span id="L-515"><a href="#L-515"><span class="linenos">515</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="L-516"><a href="#L-516"><span class="linenos">516</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="L-517"><a href="#L-517"><span class="linenos">517</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="L-518"><a href="#L-518"><span class="linenos">518</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="L-519"><a href="#L-519"><span class="linenos">519</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-520"><a href="#L-520"><span class="linenos">520</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="L-511"><a href="#L-511"><span class="linenos">511</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-512"><a href="#L-512"><span class="linenos">512</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="L-513"><a href="#L-513"><span class="linenos">513</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="L-514"><a href="#L-514"><span class="linenos">514</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="L-515"><a href="#L-515"><span class="linenos">515</span></a>
|
||||
</span><span id="L-516"><a href="#L-516"><span class="linenos">516</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-517"><a href="#L-517"><span class="linenos">517</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="L-518"><a href="#L-518"><span class="linenos">518</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-519"><a href="#L-519"><span class="linenos">519</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="L-520"><a href="#L-520"><span class="linenos">520</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-521"><a href="#L-521"><span class="linenos">521</span></a>
|
||||
</span><span id="L-522"><a href="#L-522"><span class="linenos">522</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-523"><a href="#L-523"><span class="linenos">523</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-522"><a href="#L-522"><span class="linenos">522</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-523"><a href="#L-523"><span class="linenos">523</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-524"><a href="#L-524"><span class="linenos">524</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-525"><a href="#L-525"><span class="linenos">525</span></a>
|
||||
</span><span id="L-526"><a href="#L-526"><span class="linenos">526</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="L-527"><a href="#L-527"><span class="linenos">527</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="L-528"><a href="#L-528"><span class="linenos">528</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="L-529"><a href="#L-529"><span class="linenos">529</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="L-530"><a href="#L-530"><span class="linenos">530</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-531"><a href="#L-531"><span class="linenos">531</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="L-532"><a href="#L-532"><span class="linenos">532</span></a>
|
||||
</span><span id="L-533"><a href="#L-533"><span class="linenos">533</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-534"><a href="#L-534"><span class="linenos">534</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1475,50 +1489,41 @@ For example, a generic mapping type might be defined as::</p>
|
|||
</span><span id="MappingSchema-373"><a href="#MappingSchema-373"><span class="linenos">373</span></a> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-374"><a href="#MappingSchema-374"><span class="linenos">374</span></a> <span class="n">normalize</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-375"><a href="#MappingSchema-375"><span class="linenos">375</span></a> <span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-376"><a href="#MappingSchema-376"><span class="linenos">376</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-377"><a href="#MappingSchema-377"><span class="linenos">377</span></a> <span class="n">normalize</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize</span> <span class="k">if</span> <span class="n">normalize</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">normalize</span>
|
||||
</span><span id="MappingSchema-378"><a href="#MappingSchema-378"><span class="linenos">378</span></a>
|
||||
</span><span id="MappingSchema-379"><a href="#MappingSchema-379"><span class="linenos">379</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-380"><a href="#MappingSchema-380"><span class="linenos">380</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-381"><a href="#MappingSchema-381"><span class="linenos">381</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-382"><a href="#MappingSchema-382"><span class="linenos">382</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-383"><a href="#MappingSchema-383"><span class="linenos">383</span></a>
|
||||
</span><span id="MappingSchema-384"><a href="#MappingSchema-384"><span class="linenos">384</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-385"><a href="#MappingSchema-385"><span class="linenos">385</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">normalize</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-386"><a href="#MappingSchema-386"><span class="linenos">386</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="MappingSchema-387"><a href="#MappingSchema-387"><span class="linenos">387</span></a>
|
||||
</span><span id="MappingSchema-388"><a href="#MappingSchema-388"><span class="linenos">388</span></a> <span class="c1"># This can be useful for normalize_identifier</span>
|
||||
</span><span id="MappingSchema-389"><a href="#MappingSchema-389"><span class="linenos">389</span></a> <span class="n">identifier</span><span class="o">.</span><span class="n">meta</span><span class="p">[</span><span class="s2">"is_table"</span><span class="p">]</span> <span class="o">=</span> <span class="n">is_table</span>
|
||||
</span><span id="MappingSchema-390"><a href="#MappingSchema-390"><span class="linenos">390</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-391"><a href="#MappingSchema-391"><span class="linenos">391</span></a>
|
||||
</span><span id="MappingSchema-392"><a href="#MappingSchema-392"><span class="linenos">392</span></a> <span class="k">def</span> <span class="nf">depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-393"><a href="#MappingSchema-393"><span class="linenos">393</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">empty</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-394"><a href="#MappingSchema-394"><span class="linenos">394</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="MappingSchema-395"><a href="#MappingSchema-395"><span class="linenos">395</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="MappingSchema-396"><a href="#MappingSchema-396"><span class="linenos">396</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span>
|
||||
</span><span id="MappingSchema-397"><a href="#MappingSchema-397"><span class="linenos">397</span></a>
|
||||
</span><span id="MappingSchema-398"><a href="#MappingSchema-398"><span class="linenos">398</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-399"><a href="#MappingSchema-399"><span class="linenos">399</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="MappingSchema-400"><a href="#MappingSchema-400"><span class="linenos">400</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="MappingSchema-401"><a href="#MappingSchema-401"><span class="linenos">401</span></a>
|
||||
</span><span id="MappingSchema-402"><a href="#MappingSchema-402"><span class="linenos">402</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="MappingSchema-403"><a href="#MappingSchema-403"><span class="linenos">403</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="MappingSchema-404"><a href="#MappingSchema-404"><span class="linenos">404</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="MappingSchema-405"><a href="#MappingSchema-405"><span class="linenos">405</span></a>
|
||||
</span><span id="MappingSchema-406"><a href="#MappingSchema-406"><span class="linenos">406</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="MappingSchema-407"><a href="#MappingSchema-407"><span class="linenos">407</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="MappingSchema-408"><a href="#MappingSchema-408"><span class="linenos">408</span></a><span class="sd"> """</span>
|
||||
</span><span id="MappingSchema-409"><a href="#MappingSchema-409"><span class="linenos">409</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-410"><a href="#MappingSchema-410"><span class="linenos">410</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-411"><a href="#MappingSchema-411"><span class="linenos">411</span></a>
|
||||
</span><span id="MappingSchema-412"><a href="#MappingSchema-412"><span class="linenos">412</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-413"><a href="#MappingSchema-413"><span class="linenos">413</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-414"><a href="#MappingSchema-414"><span class="linenos">414</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="MappingSchema-415"><a href="#MappingSchema-415"><span class="linenos">415</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-416"><a href="#MappingSchema-416"><span class="linenos">416</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="MappingSchema-417"><a href="#MappingSchema-417"><span class="linenos">417</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-418"><a href="#MappingSchema-418"><span class="linenos">418</span></a>
|
||||
</span><span id="MappingSchema-419"><a href="#MappingSchema-419"><span class="linenos">419</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span><span id="MappingSchema-376"><a href="#MappingSchema-376"><span class="linenos">376</span></a> <span class="k">return</span> <span class="n">normalize_name</span><span class="p">(</span>
|
||||
</span><span id="MappingSchema-377"><a href="#MappingSchema-377"><span class="linenos">377</span></a> <span class="n">name</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-378"><a href="#MappingSchema-378"><span class="linenos">378</span></a> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-379"><a href="#MappingSchema-379"><span class="linenos">379</span></a> <span class="n">is_table</span><span class="o">=</span><span class="n">is_table</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-380"><a href="#MappingSchema-380"><span class="linenos">380</span></a> <span class="n">normalize</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">normalize</span> <span class="k">if</span> <span class="n">normalize</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">normalize</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-381"><a href="#MappingSchema-381"><span class="linenos">381</span></a> <span class="p">)</span>
|
||||
</span><span id="MappingSchema-382"><a href="#MappingSchema-382"><span class="linenos">382</span></a>
|
||||
</span><span id="MappingSchema-383"><a href="#MappingSchema-383"><span class="linenos">383</span></a> <span class="k">def</span> <span class="nf">depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-384"><a href="#MappingSchema-384"><span class="linenos">384</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">empty</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-385"><a href="#MappingSchema-385"><span class="linenos">385</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="MappingSchema-386"><a href="#MappingSchema-386"><span class="linenos">386</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="MappingSchema-387"><a href="#MappingSchema-387"><span class="linenos">387</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span>
|
||||
</span><span id="MappingSchema-388"><a href="#MappingSchema-388"><span class="linenos">388</span></a>
|
||||
</span><span id="MappingSchema-389"><a href="#MappingSchema-389"><span class="linenos">389</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-390"><a href="#MappingSchema-390"><span class="linenos">390</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="MappingSchema-391"><a href="#MappingSchema-391"><span class="linenos">391</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="MappingSchema-392"><a href="#MappingSchema-392"><span class="linenos">392</span></a>
|
||||
</span><span id="MappingSchema-393"><a href="#MappingSchema-393"><span class="linenos">393</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="MappingSchema-394"><a href="#MappingSchema-394"><span class="linenos">394</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="MappingSchema-395"><a href="#MappingSchema-395"><span class="linenos">395</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="MappingSchema-396"><a href="#MappingSchema-396"><span class="linenos">396</span></a>
|
||||
</span><span id="MappingSchema-397"><a href="#MappingSchema-397"><span class="linenos">397</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="MappingSchema-398"><a href="#MappingSchema-398"><span class="linenos">398</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="MappingSchema-399"><a href="#MappingSchema-399"><span class="linenos">399</span></a><span class="sd"> """</span>
|
||||
</span><span id="MappingSchema-400"><a href="#MappingSchema-400"><span class="linenos">400</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-401"><a href="#MappingSchema-401"><span class="linenos">401</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-402"><a href="#MappingSchema-402"><span class="linenos">402</span></a>
|
||||
</span><span id="MappingSchema-403"><a href="#MappingSchema-403"><span class="linenos">403</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-404"><a href="#MappingSchema-404"><span class="linenos">404</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-405"><a href="#MappingSchema-405"><span class="linenos">405</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="MappingSchema-406"><a href="#MappingSchema-406"><span class="linenos">406</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-407"><a href="#MappingSchema-407"><span class="linenos">407</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="MappingSchema-408"><a href="#MappingSchema-408"><span class="linenos">408</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-409"><a href="#MappingSchema-409"><span class="linenos">409</span></a>
|
||||
</span><span id="MappingSchema-410"><a href="#MappingSchema-410"><span class="linenos">410</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1663,11 +1668,11 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#MappingSchema.depth"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="MappingSchema.depth-392"><a href="#MappingSchema.depth-392"><span class="linenos">392</span></a> <span class="k">def</span> <span class="nf">depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema.depth-393"><a href="#MappingSchema.depth-393"><span class="linenos">393</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">empty</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema.depth-394"><a href="#MappingSchema.depth-394"><span class="linenos">394</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="MappingSchema.depth-395"><a href="#MappingSchema.depth-395"><span class="linenos">395</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="MappingSchema.depth-396"><a href="#MappingSchema.depth-396"><span class="linenos">396</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="MappingSchema.depth-383"><a href="#MappingSchema.depth-383"><span class="linenos">383</span></a> <span class="k">def</span> <span class="nf">depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema.depth-384"><a href="#MappingSchema.depth-384"><span class="linenos">384</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">empty</span> <span class="ow">and</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema.depth-385"><a href="#MappingSchema.depth-385"><span class="linenos">385</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="MappingSchema.depth-386"><a href="#MappingSchema.depth-386"><span class="linenos">386</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="MappingSchema.depth-387"><a href="#MappingSchema.depth-387"><span class="linenos">387</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_depth</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1696,6 +1701,41 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
</div>
|
||||
</dl>
|
||||
</div>
|
||||
</section>
|
||||
<section id="normalize_name">
|
||||
<input id="normalize_name-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">normalize_name</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n"><a href="expressions.html#Identifier">sqlglot.expressions.Identifier</a></span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">normalize</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span></span><span class="return-annotation">) -> <span class="nb">str</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="normalize_name-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#normalize_name"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize_name-413"><a href="#normalize_name-413"><span class="linenos">413</span></a><span class="k">def</span> <span class="nf">normalize_name</span><span class="p">(</span>
|
||||
</span><span id="normalize_name-414"><a href="#normalize_name-414"><span class="linenos">414</span></a> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">,</span>
|
||||
</span><span id="normalize_name-415"><a href="#normalize_name-415"><span class="linenos">415</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="normalize_name-416"><a href="#normalize_name-416"><span class="linenos">416</span></a> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="normalize_name-417"><a href="#normalize_name-417"><span class="linenos">417</span></a> <span class="n">normalize</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="normalize_name-418"><a href="#normalize_name-418"><span class="linenos">418</span></a><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="normalize_name-419"><a href="#normalize_name-419"><span class="linenos">419</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="normalize_name-420"><a href="#normalize_name-420"><span class="linenos">420</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="normalize_name-421"><a href="#normalize_name-421"><span class="linenos">421</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="normalize_name-422"><a href="#normalize_name-422"><span class="linenos">422</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="normalize_name-423"><a href="#normalize_name-423"><span class="linenos">423</span></a>
|
||||
</span><span id="normalize_name-424"><a href="#normalize_name-424"><span class="linenos">424</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="normalize_name-425"><a href="#normalize_name-425"><span class="linenos">425</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">normalize</span><span class="p">:</span>
|
||||
</span><span id="normalize_name-426"><a href="#normalize_name-426"><span class="linenos">426</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="normalize_name-427"><a href="#normalize_name-427"><span class="linenos">427</span></a>
|
||||
</span><span id="normalize_name-428"><a href="#normalize_name-428"><span class="linenos">428</span></a> <span class="c1"># This can be useful for normalize_identifier</span>
|
||||
</span><span id="normalize_name-429"><a href="#normalize_name-429"><span class="linenos">429</span></a> <span class="n">identifier</span><span class="o">.</span><span class="n">meta</span><span class="p">[</span><span class="s2">"is_table"</span><span class="p">]</span> <span class="o">=</span> <span class="n">is_table</span>
|
||||
</span><span id="normalize_name-430"><a href="#normalize_name-430"><span class="linenos">430</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
||||
|
||||
</section>
|
||||
<section id="ensure_schema">
|
||||
<input id="ensure_schema-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
|
|
@ -1708,11 +1748,11 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#ensure_schema"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_schema-422"><a href="#ensure_schema-422"><span class="linenos">422</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="ensure_schema-423"><a href="#ensure_schema-423"><span class="linenos">423</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="ensure_schema-424"><a href="#ensure_schema-424"><span class="linenos">424</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="ensure_schema-425"><a href="#ensure_schema-425"><span class="linenos">425</span></a>
|
||||
</span><span id="ensure_schema-426"><a href="#ensure_schema-426"><span class="linenos">426</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_schema-433"><a href="#ensure_schema-433"><span class="linenos">433</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="ensure_schema-434"><a href="#ensure_schema-434"><span class="linenos">434</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="ensure_schema-435"><a href="#ensure_schema-435"><span class="linenos">435</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="ensure_schema-436"><a href="#ensure_schema-436"><span class="linenos">436</span></a>
|
||||
</span><span id="ensure_schema-437"><a href="#ensure_schema-437"><span class="linenos">437</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1730,24 +1770,24 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#ensure_column_mapping"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_column_mapping-429"><a href="#ensure_column_mapping-429"><span class="linenos">429</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-430"><a href="#ensure_column_mapping-430"><span class="linenos">430</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-431"><a href="#ensure_column_mapping-431"><span class="linenos">431</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="ensure_column_mapping-432"><a href="#ensure_column_mapping-432"><span class="linenos">432</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-433"><a href="#ensure_column_mapping-433"><span class="linenos">433</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="ensure_column_mapping-434"><a href="#ensure_column_mapping-434"><span class="linenos">434</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-435"><a href="#ensure_column_mapping-435"><span class="linenos">435</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="ensure_column_mapping-436"><a href="#ensure_column_mapping-436"><span class="linenos">436</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="ensure_column_mapping-437"><a href="#ensure_column_mapping-437"><span class="linenos">437</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="ensure_column_mapping-438"><a href="#ensure_column_mapping-438"><span class="linenos">438</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="ensure_column_mapping-439"><a href="#ensure_column_mapping-439"><span class="linenos">439</span></a> <span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-440"><a href="#ensure_column_mapping-440"><span class="linenos">440</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="ensure_column_mapping-441"><a href="#ensure_column_mapping-441"><span class="linenos">441</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-442"><a href="#ensure_column_mapping-442"><span class="linenos">442</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-443"><a href="#ensure_column_mapping-443"><span class="linenos">443</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-444"><a href="#ensure_column_mapping-444"><span class="linenos">444</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-445"><a href="#ensure_column_mapping-445"><span class="linenos">445</span></a>
|
||||
</span><span id="ensure_column_mapping-446"><a href="#ensure_column_mapping-446"><span class="linenos">446</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_column_mapping-440"><a href="#ensure_column_mapping-440"><span class="linenos">440</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-441"><a href="#ensure_column_mapping-441"><span class="linenos">441</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-442"><a href="#ensure_column_mapping-442"><span class="linenos">442</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="ensure_column_mapping-443"><a href="#ensure_column_mapping-443"><span class="linenos">443</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-444"><a href="#ensure_column_mapping-444"><span class="linenos">444</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="ensure_column_mapping-445"><a href="#ensure_column_mapping-445"><span class="linenos">445</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-446"><a href="#ensure_column_mapping-446"><span class="linenos">446</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="ensure_column_mapping-447"><a href="#ensure_column_mapping-447"><span class="linenos">447</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="ensure_column_mapping-448"><a href="#ensure_column_mapping-448"><span class="linenos">448</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="ensure_column_mapping-449"><a href="#ensure_column_mapping-449"><span class="linenos">449</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="ensure_column_mapping-450"><a href="#ensure_column_mapping-450"><span class="linenos">450</span></a> <span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-451"><a href="#ensure_column_mapping-451"><span class="linenos">451</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="ensure_column_mapping-452"><a href="#ensure_column_mapping-452"><span class="linenos">452</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-453"><a href="#ensure_column_mapping-453"><span class="linenos">453</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-454"><a href="#ensure_column_mapping-454"><span class="linenos">454</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-455"><a href="#ensure_column_mapping-455"><span class="linenos">455</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-456"><a href="#ensure_column_mapping-456"><span class="linenos">456</span></a>
|
||||
</span><span id="ensure_column_mapping-457"><a href="#ensure_column_mapping-457"><span class="linenos">457</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1765,19 +1805,19 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#flatten_schema"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="flatten_schema-449"><a href="#flatten_schema-449"><span class="linenos">449</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="flatten_schema-450"><a href="#flatten_schema-450"><span class="linenos">450</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="flatten_schema-451"><a href="#flatten_schema-451"><span class="linenos">451</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="flatten_schema-452"><a href="#flatten_schema-452"><span class="linenos">452</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-453"><a href="#flatten_schema-453"><span class="linenos">453</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-454"><a href="#flatten_schema-454"><span class="linenos">454</span></a>
|
||||
</span><span id="flatten_schema-455"><a href="#flatten_schema-455"><span class="linenos">455</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="flatten_schema-456"><a href="#flatten_schema-456"><span class="linenos">456</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-457"><a href="#flatten_schema-457"><span class="linenos">457</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="flatten_schema-458"><a href="#flatten_schema-458"><span class="linenos">458</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-459"><a href="#flatten_schema-459"><span class="linenos">459</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="flatten_schema-460"><a href="#flatten_schema-460"><span class="linenos">460</span></a>
|
||||
</span><span id="flatten_schema-461"><a href="#flatten_schema-461"><span class="linenos">461</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="flatten_schema-460"><a href="#flatten_schema-460"><span class="linenos">460</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="flatten_schema-461"><a href="#flatten_schema-461"><span class="linenos">461</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="flatten_schema-462"><a href="#flatten_schema-462"><span class="linenos">462</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="flatten_schema-463"><a href="#flatten_schema-463"><span class="linenos">463</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-464"><a href="#flatten_schema-464"><span class="linenos">464</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-465"><a href="#flatten_schema-465"><span class="linenos">465</span></a>
|
||||
</span><span id="flatten_schema-466"><a href="#flatten_schema-466"><span class="linenos">466</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="flatten_schema-467"><a href="#flatten_schema-467"><span class="linenos">467</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-468"><a href="#flatten_schema-468"><span class="linenos">468</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="flatten_schema-469"><a href="#flatten_schema-469"><span class="linenos">469</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-470"><a href="#flatten_schema-470"><span class="linenos">470</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="flatten_schema-471"><a href="#flatten_schema-471"><span class="linenos">471</span></a>
|
||||
</span><span id="flatten_schema-472"><a href="#flatten_schema-472"><span class="linenos">472</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1795,30 +1835,30 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#nested_get"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_get-464"><a href="#nested_get-464"><span class="linenos">464</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="nested_get-465"><a href="#nested_get-465"><span class="linenos">465</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="nested_get-466"><a href="#nested_get-466"><span class="linenos">466</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="nested_get-467"><a href="#nested_get-467"><span class="linenos">467</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_get-468"><a href="#nested_get-468"><span class="linenos">468</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="nested_get-469"><a href="#nested_get-469"><span class="linenos">469</span></a>
|
||||
</span><span id="nested_get-470"><a href="#nested_get-470"><span class="linenos">470</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_get-471"><a href="#nested_get-471"><span class="linenos">471</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="nested_get-472"><a href="#nested_get-472"><span class="linenos">472</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="nested_get-473"><a href="#nested_get-473"><span class="linenos">473</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="nested_get-474"><a href="#nested_get-474"><span class="linenos">474</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="nested_get-475"><a href="#nested_get-475"><span class="linenos">475</span></a>
|
||||
</span><span id="nested_get-476"><a href="#nested_get-476"><span class="linenos">476</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_get-477"><a href="#nested_get-477"><span class="linenos">477</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="nested_get-478"><a href="#nested_get-478"><span class="linenos">478</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_get-479"><a href="#nested_get-479"><span class="linenos">479</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="nested_get-480"><a href="#nested_get-480"><span class="linenos">480</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="nested_get-481"><a href="#nested_get-481"><span class="linenos">481</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="nested_get-482"><a href="#nested_get-482"><span class="linenos">482</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="nested_get-483"><a href="#nested_get-483"><span class="linenos">483</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="nested_get-484"><a href="#nested_get-484"><span class="linenos">484</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="nested_get-485"><a href="#nested_get-485"><span class="linenos">485</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_get-475"><a href="#nested_get-475"><span class="linenos">475</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="nested_get-476"><a href="#nested_get-476"><span class="linenos">476</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="nested_get-477"><a href="#nested_get-477"><span class="linenos">477</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="nested_get-478"><a href="#nested_get-478"><span class="linenos">478</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_get-479"><a href="#nested_get-479"><span class="linenos">479</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="nested_get-480"><a href="#nested_get-480"><span class="linenos">480</span></a>
|
||||
</span><span id="nested_get-481"><a href="#nested_get-481"><span class="linenos">481</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_get-482"><a href="#nested_get-482"><span class="linenos">482</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="nested_get-483"><a href="#nested_get-483"><span class="linenos">483</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="nested_get-484"><a href="#nested_get-484"><span class="linenos">484</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="nested_get-485"><a href="#nested_get-485"><span class="linenos">485</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="nested_get-486"><a href="#nested_get-486"><span class="linenos">486</span></a>
|
||||
</span><span id="nested_get-487"><a href="#nested_get-487"><span class="linenos">487</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_get-487"><a href="#nested_get-487"><span class="linenos">487</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_get-488"><a href="#nested_get-488"><span class="linenos">488</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="nested_get-489"><a href="#nested_get-489"><span class="linenos">489</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_get-490"><a href="#nested_get-490"><span class="linenos">490</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="nested_get-491"><a href="#nested_get-491"><span class="linenos">491</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="nested_get-492"><a href="#nested_get-492"><span class="linenos">492</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="nested_get-493"><a href="#nested_get-493"><span class="linenos">493</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="nested_get-494"><a href="#nested_get-494"><span class="linenos">494</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="nested_get-495"><a href="#nested_get-495"><span class="linenos">495</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="nested_get-496"><a href="#nested_get-496"><span class="linenos">496</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="nested_get-497"><a href="#nested_get-497"><span class="linenos">497</span></a>
|
||||
</span><span id="nested_get-498"><a href="#nested_get-498"><span class="linenos">498</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1853,41 +1893,41 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#nested_set"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_set-490"><a href="#nested_set-490"><span class="linenos">490</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="nested_set-491"><a href="#nested_set-491"><span class="linenos">491</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_set-492"><a href="#nested_set-492"><span class="linenos">492</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="nested_set-493"><a href="#nested_set-493"><span class="linenos">493</span></a>
|
||||
</span><span id="nested_set-494"><a href="#nested_set-494"><span class="linenos">494</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="nested_set-495"><a href="#nested_set-495"><span class="linenos">495</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-496"><a href="#nested_set-496"><span class="linenos">496</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-497"><a href="#nested_set-497"><span class="linenos">497</span></a>
|
||||
</span><span id="nested_set-498"><a href="#nested_set-498"><span class="linenos">498</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-499"><a href="#nested_set-499"><span class="linenos">499</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-500"><a href="#nested_set-500"><span class="linenos">500</span></a>
|
||||
</span><span id="nested_set-501"><a href="#nested_set-501"><span class="linenos">501</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_set-502"><a href="#nested_set-502"><span class="linenos">502</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="nested_set-503"><a href="#nested_set-503"><span class="linenos">503</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="nested_set-504"><a href="#nested_set-504"><span class="linenos">504</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="nested_set-505"><a href="#nested_set-505"><span class="linenos">505</span></a>
|
||||
</span><span id="nested_set-506"><a href="#nested_set-506"><span class="linenos">506</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_set-507"><a href="#nested_set-507"><span class="linenos">507</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="nested_set-508"><a href="#nested_set-508"><span class="linenos">508</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_set-509"><a href="#nested_set-509"><span class="linenos">509</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="nested_set-510"><a href="#nested_set-510"><span class="linenos">510</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_set-501"><a href="#nested_set-501"><span class="linenos">501</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="nested_set-502"><a href="#nested_set-502"><span class="linenos">502</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_set-503"><a href="#nested_set-503"><span class="linenos">503</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="nested_set-504"><a href="#nested_set-504"><span class="linenos">504</span></a>
|
||||
</span><span id="nested_set-505"><a href="#nested_set-505"><span class="linenos">505</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="nested_set-506"><a href="#nested_set-506"><span class="linenos">506</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-507"><a href="#nested_set-507"><span class="linenos">507</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-508"><a href="#nested_set-508"><span class="linenos">508</span></a>
|
||||
</span><span id="nested_set-509"><a href="#nested_set-509"><span class="linenos">509</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-510"><a href="#nested_set-510"><span class="linenos">510</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-511"><a href="#nested_set-511"><span class="linenos">511</span></a>
|
||||
</span><span id="nested_set-512"><a href="#nested_set-512"><span class="linenos">512</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nested_set-513"><a href="#nested_set-513"><span class="linenos">513</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-514"><a href="#nested_set-514"><span class="linenos">514</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-515"><a href="#nested_set-515"><span class="linenos">515</span></a>
|
||||
</span><span id="nested_set-516"><a href="#nested_set-516"><span class="linenos">516</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-517"><a href="#nested_set-517"><span class="linenos">517</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="nested_set-518"><a href="#nested_set-518"><span class="linenos">518</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="nested_set-519"><a href="#nested_set-519"><span class="linenos">519</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="nested_set-520"><a href="#nested_set-520"><span class="linenos">520</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="nested_set-521"><a href="#nested_set-521"><span class="linenos">521</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="nested_set-512"><a href="#nested_set-512"><span class="linenos">512</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_set-513"><a href="#nested_set-513"><span class="linenos">513</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="nested_set-514"><a href="#nested_set-514"><span class="linenos">514</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="nested_set-515"><a href="#nested_set-515"><span class="linenos">515</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="nested_set-516"><a href="#nested_set-516"><span class="linenos">516</span></a>
|
||||
</span><span id="nested_set-517"><a href="#nested_set-517"><span class="linenos">517</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_set-518"><a href="#nested_set-518"><span class="linenos">518</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="nested_set-519"><a href="#nested_set-519"><span class="linenos">519</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_set-520"><a href="#nested_set-520"><span class="linenos">520</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="nested_set-521"><a href="#nested_set-521"><span class="linenos">521</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-522"><a href="#nested_set-522"><span class="linenos">522</span></a>
|
||||
</span><span id="nested_set-523"><a href="#nested_set-523"><span class="linenos">523</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-524"><a href="#nested_set-524"><span class="linenos">524</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-523"><a href="#nested_set-523"><span class="linenos">523</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nested_set-524"><a href="#nested_set-524"><span class="linenos">524</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-525"><a href="#nested_set-525"><span class="linenos">525</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-526"><a href="#nested_set-526"><span class="linenos">526</span></a>
|
||||
</span><span id="nested_set-527"><a href="#nested_set-527"><span class="linenos">527</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-528"><a href="#nested_set-528"><span class="linenos">528</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="nested_set-529"><a href="#nested_set-529"><span class="linenos">529</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="nested_set-530"><a href="#nested_set-530"><span class="linenos">530</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="nested_set-531"><a href="#nested_set-531"><span class="linenos">531</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="nested_set-532"><a href="#nested_set-532"><span class="linenos">532</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="nested_set-533"><a href="#nested_set-533"><span class="linenos">533</span></a>
|
||||
</span><span id="nested_set-534"><a href="#nested_set-534"><span class="linenos">534</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-535"><a href="#nested_set-535"><span class="linenos">535</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -153,239 +153,241 @@
|
|||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="n">order</span><span class="p">:</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">window</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">order</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">window</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">row_number</span><span class="p">)</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">outer_selects</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">())</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="sa">f</span><span class="s1">'"</span><span class="si">{</span><span class="n">row_number</span><span class="si">}</span><span class="s1">" = 1'</span><span class="p">)</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">window</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">c</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">distinct_cols</span><span class="p">]))</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">window</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">row_number</span><span class="p">)</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">outer_selects</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">())</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="sa">f</span><span class="s1">'"</span><span class="si">{</span><span class="n">row_number</span><span class="si">}</span><span class="s1">" = 1'</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="k">def</span> <span class="nf">eliminate_qualify</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="sd"> Convert SELECT statements that contain the QUALIFY clause into subqueries, filtered equivalently.</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="sd"> The idea behind this transformation can be seen in Snowflake's documentation for QUALIFY:</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="sd"> https://docs.snowflake.com/en/sql-reference/constructs/qualify</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="sd"> Some dialects don't support window functions in the WHERE clause, so we need to include them as</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="sd"> projections in the subquery, in order to refer to them in the outer filter using aliases. Also,</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="sd"> if a column is referenced in the QUALIFY clause but is not selected, we need to include it too,</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="sd"> otherwise we won't be able to refer to it in the outer query's WHERE clause.</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"qualify"</span><span class="p">):</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="s2">"_c"</span><span class="p">)</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">select</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">alias</span><span class="p">))</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">taken</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">outer_selects</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">])</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"qualify"</span><span class="p">]</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">select_candidates</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span> <span class="k">if</span> <span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">for</span> <span class="n">expr</span> <span class="ow">in</span> <span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">):</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">):</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">,</span> <span class="s2">"_w"</span><span class="p">)</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">alias</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">column</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Qualify</span><span class="p">):</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">column</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">expr</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">elif</span> <span class="n">expr</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">return</span> <span class="n">outer_selects</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">alias</span><span class="o">=</span><span class="s2">"_t"</span><span class="p">))</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">qualify_filters</span><span class="p">)</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="k">def</span> <span class="nf">eliminate_qualify</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="sd"> Convert SELECT statements that contain the QUALIFY clause into subqueries, filtered equivalently.</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a><span class="sd"> The idea behind this transformation can be seen in Snowflake's documentation for QUALIFY:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="sd"> https://docs.snowflake.com/en/sql-reference/constructs/qualify</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="sd"> Some dialects don't support window functions in the WHERE clause, so we need to include them as</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="sd"> projections in the subquery, in order to refer to them in the outer filter using aliases. Also,</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="sd"> if a column is referenced in the QUALIFY clause but is not selected, we need to include it too,</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a><span class="sd"> otherwise we won't be able to refer to it in the outer query's WHERE clause.</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"qualify"</span><span class="p">):</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="s2">"_c"</span><span class="p">)</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">select</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">alias</span><span class="p">))</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">taken</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">outer_selects</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">])</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"qualify"</span><span class="p">]</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">select_candidates</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span> <span class="k">if</span> <span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">for</span> <span class="n">expr</span> <span class="ow">in</span> <span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">):</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">):</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">,</span> <span class="s2">"_w"</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">alias</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">column</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Qualify</span><span class="p">):</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">column</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">expr</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">elif</span> <span class="n">expr</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">return</span> <span class="n">outer_selects</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">alias</span><span class="o">=</span><span class="s2">"_t"</span><span class="p">))</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">qualify_filters</span><span class="p">)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a><span class="k">def</span> <span class="nf">remove_precision_parameterized_types</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a><span class="sd"> Some dialects only allow the precision for parameterized types to be defined in the DDL and not in</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a><span class="sd"> other expressions. This transforms removes the precision from parameterized types in expressions.</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">):</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">node</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="s2">"expressions"</span><span class="p">,</span> <span class="p">[</span><span class="n">e</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataTypeSize</span><span class="p">)]</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="p">)</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a><span class="k">def</span> <span class="nf">remove_precision_parameterized_types</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a><span class="sd"> Some dialects only allow the precision for parameterized types to be defined in the DDL and not in</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a><span class="sd"> other expressions. This transforms removes the precision from parameterized types in expressions.</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">):</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">node</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="s2">"expressions"</span><span class="p">,</span> <span class="p">[</span><span class="n">e</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataTypeParam</span><span class="p">)]</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="p">)</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a><span class="k">def</span> <span class="nf">unnest_to_explode</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a><span class="w"> </span><span class="sd">"""Convert cross join unnest into lateral view explode (used in presto -> hive)."""</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">):</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span> <span class="k">if</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"ordinality"</span><span class="p">)</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Explode</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a><span class="k">def</span> <span class="nf">unnest_to_explode</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a><span class="w"> </span><span class="sd">"""Convert cross join unnest into lateral view explode (used in presto -> hive)."""</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">):</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span> <span class="k">if</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"ordinality"</span><span class="p">)</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Explode</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">for</span> <span class="n">e</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">unnest</span><span class="o">.</span><span class="n">expressions</span><span class="p">,</span> <span class="n">alias</span><span class="o">.</span><span class="n">columns</span> <span class="k">if</span> <span class="n">alias</span> <span class="k">else</span> <span class="p">[]):</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="s2">"laterals"</span><span class="p">,</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">(</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">this</span><span class="o">=</span><span class="n">udtf</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">e</span><span class="p">),</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">view</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">alias</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">column</span><span class="p">]),</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="p">),</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="p">)</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">for</span> <span class="n">e</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">unnest</span><span class="o">.</span><span class="n">expressions</span><span class="p">,</span> <span class="n">alias</span><span class="o">.</span><span class="n">columns</span> <span class="k">if</span> <span class="n">alias</span> <span class="k">else</span> <span class="p">[]):</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="s2">"laterals"</span><span class="p">,</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">(</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">this</span><span class="o">=</span><span class="n">udtf</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">e</span><span class="p">),</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">view</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">alias</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">column</span><span class="p">]),</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="p">),</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="p">)</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a><span class="k">def</span> <span class="nf">explode_to_unnest</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a><span class="w"> </span><span class="sd">"""Convert explode/posexplode into unnest (used in hive -> presto)."""</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">taken_select_names</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">taken_source_names</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">Scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">references</span><span class="p">}</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Aliases</span><span class="p">):</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Explode</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)):</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">is_posexplode</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="n">explode_arg</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">copy</span><span class="p">()],</span> <span class="n">ordinality</span><span class="o">=</span><span class="n">is_posexplode</span><span class="p">)</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="c1"># This ensures that we won't use [POS]EXPLODE's argument as a new selection</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">explode_arg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">output_name</span><span class="p">)</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="n">unnest_source_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_source_names</span><span class="p">,</span> <span class="s2">"_u"</span><span class="p">)</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">taken_source_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">unnest_source_alias</span><span class="p">)</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">explode_alias</span><span class="p">:</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"col"</span><span class="p">)</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">)</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"pos"</span><span class="p">)</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">)</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">,</span> <span class="n">pos_alias</span><span class="p">]</span>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">,</span> <span class="n">explode_alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">]</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">))</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">unnest_source_alias</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">column_names</span><span class="p">)</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a><span class="k">def</span> <span class="nf">explode_to_unnest</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a><span class="w"> </span><span class="sd">"""Convert explode/posexplode into unnest (used in hive -> presto)."""</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="n">taken_select_names</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">taken_source_names</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">Scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">references</span><span class="p">}</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Aliases</span><span class="p">):</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Explode</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)):</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="n">is_posexplode</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">explode_arg</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">copy</span><span class="p">()],</span> <span class="n">ordinality</span><span class="o">=</span><span class="n">is_posexplode</span><span class="p">)</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="c1"># This ensures that we won't use [POS]EXPLODE's argument as a new selection</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">explode_arg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">output_name</span><span class="p">)</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">unnest_source_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_source_names</span><span class="p">,</span> <span class="s2">"_u"</span><span class="p">)</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">taken_source_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">unnest_source_alias</span><span class="p">)</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">explode_alias</span><span class="p">:</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"col"</span><span class="p">)</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">)</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"pos"</span><span class="p">)</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">)</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">,</span> <span class="n">pos_alias</span><span class="p">]</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">,</span> <span class="n">explode_alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">]</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">))</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">):</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">join_type</span><span class="o">=</span><span class="s2">"CROSS"</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">unnest_source_alias</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">column_names</span><span class="p">)</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">):</span>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">join_type</span><span class="o">=</span><span class="s2">"CROSS"</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a><span class="k">def</span> <span class="nf">remove_within_group_for_percentiles</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">)</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">PercentileCont</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">PercentileDisc</span><span class="p">))</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="p">):</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="n">quantile</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="n">input_value</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">))</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">ApproxQuantile</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">input_value</span><span class="p">,</span> <span class="n">quantile</span><span class="o">=</span><span class="n">quantile</span><span class="p">))</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a><span class="k">def</span> <span class="nf">remove_within_group_for_percentiles</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">)</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">PercentileCont</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">PercentileDisc</span><span class="p">))</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="p">):</span>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="n">quantile</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">input_value</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">))</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">ApproxQuantile</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">input_value</span><span class="p">,</span> <span class="n">quantile</span><span class="o">=</span><span class="n">quantile</span><span class="p">))</span>
|
||||
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a><span class="k">def</span> <span class="nf">add_recursive_cte_column_names</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="n">next_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_c_"</span><span class="p">)</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="k">for</span> <span class="n">cte</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">query</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="s2">"columns"</span><span class="p">,</span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">or</span> <span class="n">next_name</span><span class="p">())</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">query</span><span class="o">.</span><span class="n">selects</span><span class="p">],</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="p">)</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a><span class="k">def</span> <span class="nf">add_recursive_cte_column_names</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="n">next_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_c_"</span><span class="p">)</span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="k">for</span> <span class="n">cte</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">query</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="s2">"columns"</span><span class="p">,</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">or</span> <span class="n">next_name</span><span class="p">())</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">query</span><span class="o">.</span><span class="n">selects</span><span class="p">],</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="p">)</span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a><span class="k">def</span> <span class="nf">epoch_cast_to_ts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TryCast</span><span class="p">))</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"epoch"</span>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="p">):</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="o">.</span><span class="n">string</span><span class="p">(</span><span class="s2">"1970-01-01 00:00:00"</span><span class="p">))</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a><span class="k">def</span> <span class="nf">epoch_cast_to_ts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TryCast</span><span class="p">))</span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"epoch"</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="p">):</span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="o">.</span><span class="n">string</span><span class="p">(</span><span class="s2">"1970-01-01 00:00:00"</span><span class="p">))</span>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a><span class="k">def</span> <span class="nf">preprocess</span><span class="p">(</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="n">transforms</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]],</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">Generator</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a><span class="sd"> Creates a new transform by chaining a sequence of transformations and converts the resulting</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a><span class="sd"> expression to SQL, using either the "_sql" method corresponding to the resulting expression,</span>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a><span class="sd"> or the appropriate `Generator.TRANSFORMS` function (when applicable -- see below).</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a><span class="sd"> transforms: sequence of transform functions. These will be called in order.</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a><span class="sd"> Function that can be used as a generator transform.</span>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="k">def</span> <span class="nf">_to_sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="n">expression_type</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">0</span><span class="p">](</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">t</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a>
|
||||
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a> <span class="n">_sql_handler</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">key</span> <span class="o">+</span> <span class="s2">"_sql"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="k">if</span> <span class="n">_sql_handler</span><span class="p">:</span>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a> <span class="k">return</span> <span class="n">_sql_handler</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a> <span class="n">transforms_handler</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">TRANSFORMS</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">))</span>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="k">if</span> <span class="n">transforms_handler</span><span class="p">:</span>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="c1"># Ensures we don't enter an infinite loop. This can happen when the original expression</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="c1"># has the same type as the final expression and there's no _sql method available for it,</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="c1"># because then it'd re-enter _to_sql.</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="k">if</span> <span class="n">expression_type</span> <span class="ow">is</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="sa">f</span><span class="s2">"Expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2"> requires a _sql method in order to be transformed."</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="p">)</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="k">return</span> <span class="n">transforms_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a><span class="k">def</span> <span class="nf">preprocess</span><span class="p">(</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="n">transforms</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]],</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">Generator</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a><span class="sd"> Creates a new transform by chaining a sequence of transformations and converts the resulting</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a><span class="sd"> expression to SQL, using either the "_sql" method corresponding to the resulting expression,</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a><span class="sd"> or the appropriate `Generator.TRANSFORMS` function (when applicable -- see below).</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a><span class="sd"> transforms: sequence of transform functions. These will be called in order.</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a><span class="sd"> Function that can be used as a generator transform.</span>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="k">def</span> <span class="nf">_to_sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">expression_type</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">0</span><span class="p">](</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">t</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a> <span class="n">_sql_handler</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">key</span> <span class="o">+</span> <span class="s2">"_sql"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a> <span class="k">if</span> <span class="n">_sql_handler</span><span class="p">:</span>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a> <span class="k">return</span> <span class="n">_sql_handler</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="n">transforms_handler</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">TRANSFORMS</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">))</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="k">if</span> <span class="n">transforms_handler</span><span class="p">:</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="c1"># Ensures we don't enter an infinite loop. This can happen when the original expression</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="c1"># has the same type as the final expression and there's no _sql method available for it,</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="c1"># because then it'd re-enter _to_sql.</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="k">if</span> <span class="n">expression_type</span> <span class="ow">is</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="sa">f</span><span class="s2">"Expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2"> requires a _sql method in order to be transformed."</span>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="p">)</span>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unsupported expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="k">return</span> <span class="n">transforms_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">return</span> <span class="n">_to_sql</span>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unsupported expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="k">return</span> <span class="n">_to_sql</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -500,13 +502,15 @@
|
|||
</span><span id="eliminate_distinct_on-69"><a href="#eliminate_distinct_on-69"><span class="linenos">69</span></a>
|
||||
</span><span id="eliminate_distinct_on-70"><a href="#eliminate_distinct_on-70"><span class="linenos">70</span></a> <span class="k">if</span> <span class="n">order</span><span class="p">:</span>
|
||||
</span><span id="eliminate_distinct_on-71"><a href="#eliminate_distinct_on-71"><span class="linenos">71</span></a> <span class="n">window</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">order</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="eliminate_distinct_on-72"><a href="#eliminate_distinct_on-72"><span class="linenos">72</span></a>
|
||||
</span><span id="eliminate_distinct_on-73"><a href="#eliminate_distinct_on-73"><span class="linenos">73</span></a> <span class="n">window</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">row_number</span><span class="p">)</span>
|
||||
</span><span id="eliminate_distinct_on-74"><a href="#eliminate_distinct_on-74"><span class="linenos">74</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="eliminate_distinct_on-75"><a href="#eliminate_distinct_on-75"><span class="linenos">75</span></a>
|
||||
</span><span id="eliminate_distinct_on-76"><a href="#eliminate_distinct_on-76"><span class="linenos">76</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">outer_selects</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">())</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="sa">f</span><span class="s1">'"</span><span class="si">{</span><span class="n">row_number</span><span class="si">}</span><span class="s1">" = 1'</span><span class="p">)</span>
|
||||
</span><span id="eliminate_distinct_on-72"><a href="#eliminate_distinct_on-72"><span class="linenos">72</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="eliminate_distinct_on-73"><a href="#eliminate_distinct_on-73"><span class="linenos">73</span></a> <span class="n">window</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">c</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">distinct_cols</span><span class="p">]))</span>
|
||||
</span><span id="eliminate_distinct_on-74"><a href="#eliminate_distinct_on-74"><span class="linenos">74</span></a>
|
||||
</span><span id="eliminate_distinct_on-75"><a href="#eliminate_distinct_on-75"><span class="linenos">75</span></a> <span class="n">window</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">row_number</span><span class="p">)</span>
|
||||
</span><span id="eliminate_distinct_on-76"><a href="#eliminate_distinct_on-76"><span class="linenos">76</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">window</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="eliminate_distinct_on-77"><a href="#eliminate_distinct_on-77"><span class="linenos">77</span></a>
|
||||
</span><span id="eliminate_distinct_on-78"><a href="#eliminate_distinct_on-78"><span class="linenos">78</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="eliminate_distinct_on-78"><a href="#eliminate_distinct_on-78"><span class="linenos">78</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">outer_selects</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">())</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="sa">f</span><span class="s1">'"</span><span class="si">{</span><span class="n">row_number</span><span class="si">}</span><span class="s1">" = 1'</span><span class="p">)</span>
|
||||
</span><span id="eliminate_distinct_on-79"><a href="#eliminate_distinct_on-79"><span class="linenos">79</span></a>
|
||||
</span><span id="eliminate_distinct_on-80"><a href="#eliminate_distinct_on-80"><span class="linenos">80</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -540,46 +544,46 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#eliminate_qualify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="eliminate_qualify-81"><a href="#eliminate_qualify-81"><span class="linenos"> 81</span></a><span class="k">def</span> <span class="nf">eliminate_qualify</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-82"><a href="#eliminate_qualify-82"><span class="linenos"> 82</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="eliminate_qualify-83"><a href="#eliminate_qualify-83"><span class="linenos"> 83</span></a><span class="sd"> Convert SELECT statements that contain the QUALIFY clause into subqueries, filtered equivalently.</span>
|
||||
</span><span id="eliminate_qualify-84"><a href="#eliminate_qualify-84"><span class="linenos"> 84</span></a>
|
||||
</span><span id="eliminate_qualify-85"><a href="#eliminate_qualify-85"><span class="linenos"> 85</span></a><span class="sd"> The idea behind this transformation can be seen in Snowflake's documentation for QUALIFY:</span>
|
||||
</span><span id="eliminate_qualify-86"><a href="#eliminate_qualify-86"><span class="linenos"> 86</span></a><span class="sd"> https://docs.snowflake.com/en/sql-reference/constructs/qualify</span>
|
||||
</span><span id="eliminate_qualify-87"><a href="#eliminate_qualify-87"><span class="linenos"> 87</span></a>
|
||||
</span><span id="eliminate_qualify-88"><a href="#eliminate_qualify-88"><span class="linenos"> 88</span></a><span class="sd"> Some dialects don't support window functions in the WHERE clause, so we need to include them as</span>
|
||||
</span><span id="eliminate_qualify-89"><a href="#eliminate_qualify-89"><span class="linenos"> 89</span></a><span class="sd"> projections in the subquery, in order to refer to them in the outer filter using aliases. Also,</span>
|
||||
</span><span id="eliminate_qualify-90"><a href="#eliminate_qualify-90"><span class="linenos"> 90</span></a><span class="sd"> if a column is referenced in the QUALIFY clause but is not selected, we need to include it too,</span>
|
||||
</span><span id="eliminate_qualify-91"><a href="#eliminate_qualify-91"><span class="linenos"> 91</span></a><span class="sd"> otherwise we won't be able to refer to it in the outer query's WHERE clause.</span>
|
||||
</span><span id="eliminate_qualify-92"><a href="#eliminate_qualify-92"><span class="linenos"> 92</span></a><span class="sd"> """</span>
|
||||
</span><span id="eliminate_qualify-93"><a href="#eliminate_qualify-93"><span class="linenos"> 93</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"qualify"</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-94"><a href="#eliminate_qualify-94"><span class="linenos"> 94</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-95"><a href="#eliminate_qualify-95"><span class="linenos"> 95</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-96"><a href="#eliminate_qualify-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-97"><a href="#eliminate_qualify-97"><span class="linenos"> 97</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="s2">"_c"</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-98"><a href="#eliminate_qualify-98"><span class="linenos"> 98</span></a> <span class="n">select</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">alias</span><span class="p">))</span>
|
||||
</span><span id="eliminate_qualify-99"><a href="#eliminate_qualify-99"><span class="linenos"> 99</span></a> <span class="n">taken</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-100"><a href="#eliminate_qualify-100"><span class="linenos">100</span></a>
|
||||
</span><span id="eliminate_qualify-101"><a href="#eliminate_qualify-101"><span class="linenos">101</span></a> <span class="n">outer_selects</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">])</span>
|
||||
</span><span id="eliminate_qualify-102"><a href="#eliminate_qualify-102"><span class="linenos">102</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"qualify"</span><span class="p">]</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="eliminate_qualify-103"><a href="#eliminate_qualify-103"><span class="linenos">103</span></a>
|
||||
</span><span id="eliminate_qualify-104"><a href="#eliminate_qualify-104"><span class="linenos">104</span></a> <span class="n">select_candidates</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span> <span class="k">if</span> <span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-105"><a href="#eliminate_qualify-105"><span class="linenos">105</span></a> <span class="k">for</span> <span class="n">expr</span> <span class="ow">in</span> <span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-106"><a href="#eliminate_qualify-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-107"><a href="#eliminate_qualify-107"><span class="linenos">107</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">,</span> <span class="s2">"_w"</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-108"><a href="#eliminate_qualify-108"><span class="linenos">108</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">alias</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-109"><a href="#eliminate_qualify-109"><span class="linenos">109</span></a> <span class="n">column</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-110"><a href="#eliminate_qualify-110"><span class="linenos">110</span></a>
|
||||
</span><span id="eliminate_qualify-111"><a href="#eliminate_qualify-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Qualify</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-112"><a href="#eliminate_qualify-112"><span class="linenos">112</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">column</span>
|
||||
</span><span id="eliminate_qualify-113"><a href="#eliminate_qualify-113"><span class="linenos">113</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-114"><a href="#eliminate_qualify-114"><span class="linenos">114</span></a> <span class="n">expr</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-115"><a href="#eliminate_qualify-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="n">expr</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-116"><a href="#eliminate_qualify-116"><span class="linenos">116</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-117"><a href="#eliminate_qualify-117"><span class="linenos">117</span></a>
|
||||
</span><span id="eliminate_qualify-118"><a href="#eliminate_qualify-118"><span class="linenos">118</span></a> <span class="k">return</span> <span class="n">outer_selects</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">alias</span><span class="o">=</span><span class="s2">"_t"</span><span class="p">))</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">qualify_filters</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="eliminate_qualify-83"><a href="#eliminate_qualify-83"><span class="linenos"> 83</span></a><span class="k">def</span> <span class="nf">eliminate_qualify</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-84"><a href="#eliminate_qualify-84"><span class="linenos"> 84</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="eliminate_qualify-85"><a href="#eliminate_qualify-85"><span class="linenos"> 85</span></a><span class="sd"> Convert SELECT statements that contain the QUALIFY clause into subqueries, filtered equivalently.</span>
|
||||
</span><span id="eliminate_qualify-86"><a href="#eliminate_qualify-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="eliminate_qualify-87"><a href="#eliminate_qualify-87"><span class="linenos"> 87</span></a><span class="sd"> The idea behind this transformation can be seen in Snowflake's documentation for QUALIFY:</span>
|
||||
</span><span id="eliminate_qualify-88"><a href="#eliminate_qualify-88"><span class="linenos"> 88</span></a><span class="sd"> https://docs.snowflake.com/en/sql-reference/constructs/qualify</span>
|
||||
</span><span id="eliminate_qualify-89"><a href="#eliminate_qualify-89"><span class="linenos"> 89</span></a>
|
||||
</span><span id="eliminate_qualify-90"><a href="#eliminate_qualify-90"><span class="linenos"> 90</span></a><span class="sd"> Some dialects don't support window functions in the WHERE clause, so we need to include them as</span>
|
||||
</span><span id="eliminate_qualify-91"><a href="#eliminate_qualify-91"><span class="linenos"> 91</span></a><span class="sd"> projections in the subquery, in order to refer to them in the outer filter using aliases. Also,</span>
|
||||
</span><span id="eliminate_qualify-92"><a href="#eliminate_qualify-92"><span class="linenos"> 92</span></a><span class="sd"> if a column is referenced in the QUALIFY clause but is not selected, we need to include it too,</span>
|
||||
</span><span id="eliminate_qualify-93"><a href="#eliminate_qualify-93"><span class="linenos"> 93</span></a><span class="sd"> otherwise we won't be able to refer to it in the outer query's WHERE clause.</span>
|
||||
</span><span id="eliminate_qualify-94"><a href="#eliminate_qualify-94"><span class="linenos"> 94</span></a><span class="sd"> """</span>
|
||||
</span><span id="eliminate_qualify-95"><a href="#eliminate_qualify-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"qualify"</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-96"><a href="#eliminate_qualify-96"><span class="linenos"> 96</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-97"><a href="#eliminate_qualify-97"><span class="linenos"> 97</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-98"><a href="#eliminate_qualify-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-99"><a href="#eliminate_qualify-99"><span class="linenos"> 99</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="s2">"_c"</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-100"><a href="#eliminate_qualify-100"><span class="linenos">100</span></a> <span class="n">select</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">alias</span><span class="p">))</span>
|
||||
</span><span id="eliminate_qualify-101"><a href="#eliminate_qualify-101"><span class="linenos">101</span></a> <span class="n">taken</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-102"><a href="#eliminate_qualify-102"><span class="linenos">102</span></a>
|
||||
</span><span id="eliminate_qualify-103"><a href="#eliminate_qualify-103"><span class="linenos">103</span></a> <span class="n">outer_selects</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">])</span>
|
||||
</span><span id="eliminate_qualify-104"><a href="#eliminate_qualify-104"><span class="linenos">104</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"qualify"</span><span class="p">]</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="eliminate_qualify-105"><a href="#eliminate_qualify-105"><span class="linenos">105</span></a>
|
||||
</span><span id="eliminate_qualify-106"><a href="#eliminate_qualify-106"><span class="linenos">106</span></a> <span class="n">select_candidates</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span> <span class="k">if</span> <span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-107"><a href="#eliminate_qualify-107"><span class="linenos">107</span></a> <span class="k">for</span> <span class="n">expr</span> <span class="ow">in</span> <span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-108"><a href="#eliminate_qualify-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-109"><a href="#eliminate_qualify-109"><span class="linenos">109</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">,</span> <span class="s2">"_w"</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-110"><a href="#eliminate_qualify-110"><span class="linenos">110</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">expr</span><span class="p">,</span> <span class="n">alias</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-111"><a href="#eliminate_qualify-111"><span class="linenos">111</span></a> <span class="n">column</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-112"><a href="#eliminate_qualify-112"><span class="linenos">112</span></a>
|
||||
</span><span id="eliminate_qualify-113"><a href="#eliminate_qualify-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Qualify</span><span class="p">):</span>
|
||||
</span><span id="eliminate_qualify-114"><a href="#eliminate_qualify-114"><span class="linenos">114</span></a> <span class="n">qualify_filters</span> <span class="o">=</span> <span class="n">column</span>
|
||||
</span><span id="eliminate_qualify-115"><a href="#eliminate_qualify-115"><span class="linenos">115</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-116"><a href="#eliminate_qualify-116"><span class="linenos">116</span></a> <span class="n">expr</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-117"><a href="#eliminate_qualify-117"><span class="linenos">117</span></a> <span class="k">elif</span> <span class="n">expr</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span>
|
||||
</span><span id="eliminate_qualify-118"><a href="#eliminate_qualify-118"><span class="linenos">118</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">expr</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-119"><a href="#eliminate_qualify-119"><span class="linenos">119</span></a>
|
||||
</span><span id="eliminate_qualify-120"><a href="#eliminate_qualify-120"><span class="linenos">120</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="eliminate_qualify-120"><a href="#eliminate_qualify-120"><span class="linenos">120</span></a> <span class="k">return</span> <span class="n">outer_selects</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">alias</span><span class="o">=</span><span class="s2">"_t"</span><span class="p">))</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">qualify_filters</span><span class="p">)</span>
|
||||
</span><span id="eliminate_qualify-121"><a href="#eliminate_qualify-121"><span class="linenos">121</span></a>
|
||||
</span><span id="eliminate_qualify-122"><a href="#eliminate_qualify-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -607,17 +611,17 @@ otherwise we won't be able to refer to it in the outer query's WHERE clause.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#remove_precision_parameterized_types"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_precision_parameterized_types-123"><a href="#remove_precision_parameterized_types-123"><span class="linenos">123</span></a><span class="k">def</span> <span class="nf">remove_precision_parameterized_types</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_precision_parameterized_types-124"><a href="#remove_precision_parameterized_types-124"><span class="linenos">124</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="remove_precision_parameterized_types-125"><a href="#remove_precision_parameterized_types-125"><span class="linenos">125</span></a><span class="sd"> Some dialects only allow the precision for parameterized types to be defined in the DDL and not in</span>
|
||||
</span><span id="remove_precision_parameterized_types-126"><a href="#remove_precision_parameterized_types-126"><span class="linenos">126</span></a><span class="sd"> other expressions. This transforms removes the precision from parameterized types in expressions.</span>
|
||||
</span><span id="remove_precision_parameterized_types-127"><a href="#remove_precision_parameterized_types-127"><span class="linenos">127</span></a><span class="sd"> """</span>
|
||||
</span><span id="remove_precision_parameterized_types-128"><a href="#remove_precision_parameterized_types-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">):</span>
|
||||
</span><span id="remove_precision_parameterized_types-129"><a href="#remove_precision_parameterized_types-129"><span class="linenos">129</span></a> <span class="n">node</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="remove_precision_parameterized_types-130"><a href="#remove_precision_parameterized_types-130"><span class="linenos">130</span></a> <span class="s2">"expressions"</span><span class="p">,</span> <span class="p">[</span><span class="n">e</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataTypeSize</span><span class="p">)]</span>
|
||||
</span><span id="remove_precision_parameterized_types-131"><a href="#remove_precision_parameterized_types-131"><span class="linenos">131</span></a> <span class="p">)</span>
|
||||
</span><span id="remove_precision_parameterized_types-132"><a href="#remove_precision_parameterized_types-132"><span class="linenos">132</span></a>
|
||||
</span><span id="remove_precision_parameterized_types-133"><a href="#remove_precision_parameterized_types-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_precision_parameterized_types-125"><a href="#remove_precision_parameterized_types-125"><span class="linenos">125</span></a><span class="k">def</span> <span class="nf">remove_precision_parameterized_types</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_precision_parameterized_types-126"><a href="#remove_precision_parameterized_types-126"><span class="linenos">126</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="remove_precision_parameterized_types-127"><a href="#remove_precision_parameterized_types-127"><span class="linenos">127</span></a><span class="sd"> Some dialects only allow the precision for parameterized types to be defined in the DDL and not in</span>
|
||||
</span><span id="remove_precision_parameterized_types-128"><a href="#remove_precision_parameterized_types-128"><span class="linenos">128</span></a><span class="sd"> other expressions. This transforms removes the precision from parameterized types in expressions.</span>
|
||||
</span><span id="remove_precision_parameterized_types-129"><a href="#remove_precision_parameterized_types-129"><span class="linenos">129</span></a><span class="sd"> """</span>
|
||||
</span><span id="remove_precision_parameterized_types-130"><a href="#remove_precision_parameterized_types-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">):</span>
|
||||
</span><span id="remove_precision_parameterized_types-131"><a href="#remove_precision_parameterized_types-131"><span class="linenos">131</span></a> <span class="n">node</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="remove_precision_parameterized_types-132"><a href="#remove_precision_parameterized_types-132"><span class="linenos">132</span></a> <span class="s2">"expressions"</span><span class="p">,</span> <span class="p">[</span><span class="n">e</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataTypeParam</span><span class="p">)]</span>
|
||||
</span><span id="remove_precision_parameterized_types-133"><a href="#remove_precision_parameterized_types-133"><span class="linenos">133</span></a> <span class="p">)</span>
|
||||
</span><span id="remove_precision_parameterized_types-134"><a href="#remove_precision_parameterized_types-134"><span class="linenos">134</span></a>
|
||||
</span><span id="remove_precision_parameterized_types-135"><a href="#remove_precision_parameterized_types-135"><span class="linenos">135</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -638,29 +642,29 @@ other expressions. This transforms removes the precision from parameterized type
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#unnest_to_explode"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="unnest_to_explode-136"><a href="#unnest_to_explode-136"><span class="linenos">136</span></a><span class="k">def</span> <span class="nf">unnest_to_explode</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="unnest_to_explode-137"><a href="#unnest_to_explode-137"><span class="linenos">137</span></a><span class="w"> </span><span class="sd">"""Convert cross join unnest into lateral view explode (used in presto -> hive)."""</span>
|
||||
</span><span id="unnest_to_explode-138"><a href="#unnest_to_explode-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="unnest_to_explode-139"><a href="#unnest_to_explode-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="unnest_to_explode-140"><a href="#unnest_to_explode-140"><span class="linenos">140</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="unnest_to_explode-141"><a href="#unnest_to_explode-141"><span class="linenos">141</span></a>
|
||||
</span><span id="unnest_to_explode-142"><a href="#unnest_to_explode-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">):</span>
|
||||
</span><span id="unnest_to_explode-143"><a href="#unnest_to_explode-143"><span class="linenos">143</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="unnest_to_explode-144"><a href="#unnest_to_explode-144"><span class="linenos">144</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span> <span class="k">if</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"ordinality"</span><span class="p">)</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Explode</span>
|
||||
</span><span id="unnest_to_explode-145"><a href="#unnest_to_explode-145"><span class="linenos">145</span></a>
|
||||
</span><span id="unnest_to_explode-146"><a href="#unnest_to_explode-146"><span class="linenos">146</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="unnest_to_explode-138"><a href="#unnest_to_explode-138"><span class="linenos">138</span></a><span class="k">def</span> <span class="nf">unnest_to_explode</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="unnest_to_explode-139"><a href="#unnest_to_explode-139"><span class="linenos">139</span></a><span class="w"> </span><span class="sd">"""Convert cross join unnest into lateral view explode (used in presto -> hive)."""</span>
|
||||
</span><span id="unnest_to_explode-140"><a href="#unnest_to_explode-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="unnest_to_explode-141"><a href="#unnest_to_explode-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="unnest_to_explode-142"><a href="#unnest_to_explode-142"><span class="linenos">142</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="unnest_to_explode-143"><a href="#unnest_to_explode-143"><span class="linenos">143</span></a>
|
||||
</span><span id="unnest_to_explode-144"><a href="#unnest_to_explode-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">):</span>
|
||||
</span><span id="unnest_to_explode-145"><a href="#unnest_to_explode-145"><span class="linenos">145</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="unnest_to_explode-146"><a href="#unnest_to_explode-146"><span class="linenos">146</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span> <span class="k">if</span> <span class="n">unnest</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"ordinality"</span><span class="p">)</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Explode</span>
|
||||
</span><span id="unnest_to_explode-147"><a href="#unnest_to_explode-147"><span class="linenos">147</span></a>
|
||||
</span><span id="unnest_to_explode-148"><a href="#unnest_to_explode-148"><span class="linenos">148</span></a> <span class="k">for</span> <span class="n">e</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">unnest</span><span class="o">.</span><span class="n">expressions</span><span class="p">,</span> <span class="n">alias</span><span class="o">.</span><span class="n">columns</span> <span class="k">if</span> <span class="n">alias</span> <span class="k">else</span> <span class="p">[]):</span>
|
||||
</span><span id="unnest_to_explode-149"><a href="#unnest_to_explode-149"><span class="linenos">149</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="unnest_to_explode-150"><a href="#unnest_to_explode-150"><span class="linenos">150</span></a> <span class="s2">"laterals"</span><span class="p">,</span>
|
||||
</span><span id="unnest_to_explode-151"><a href="#unnest_to_explode-151"><span class="linenos">151</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">(</span>
|
||||
</span><span id="unnest_to_explode-152"><a href="#unnest_to_explode-152"><span class="linenos">152</span></a> <span class="n">this</span><span class="o">=</span><span class="n">udtf</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">e</span><span class="p">),</span>
|
||||
</span><span id="unnest_to_explode-153"><a href="#unnest_to_explode-153"><span class="linenos">153</span></a> <span class="n">view</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="unnest_to_explode-154"><a href="#unnest_to_explode-154"><span class="linenos">154</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">alias</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">column</span><span class="p">]),</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="unnest_to_explode-155"><a href="#unnest_to_explode-155"><span class="linenos">155</span></a> <span class="p">),</span>
|
||||
</span><span id="unnest_to_explode-156"><a href="#unnest_to_explode-156"><span class="linenos">156</span></a> <span class="p">)</span>
|
||||
</span><span id="unnest_to_explode-157"><a href="#unnest_to_explode-157"><span class="linenos">157</span></a>
|
||||
</span><span id="unnest_to_explode-158"><a href="#unnest_to_explode-158"><span class="linenos">158</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="unnest_to_explode-148"><a href="#unnest_to_explode-148"><span class="linenos">148</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="unnest_to_explode-149"><a href="#unnest_to_explode-149"><span class="linenos">149</span></a>
|
||||
</span><span id="unnest_to_explode-150"><a href="#unnest_to_explode-150"><span class="linenos">150</span></a> <span class="k">for</span> <span class="n">e</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">unnest</span><span class="o">.</span><span class="n">expressions</span><span class="p">,</span> <span class="n">alias</span><span class="o">.</span><span class="n">columns</span> <span class="k">if</span> <span class="n">alias</span> <span class="k">else</span> <span class="p">[]):</span>
|
||||
</span><span id="unnest_to_explode-151"><a href="#unnest_to_explode-151"><span class="linenos">151</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="unnest_to_explode-152"><a href="#unnest_to_explode-152"><span class="linenos">152</span></a> <span class="s2">"laterals"</span><span class="p">,</span>
|
||||
</span><span id="unnest_to_explode-153"><a href="#unnest_to_explode-153"><span class="linenos">153</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Lateral</span><span class="p">(</span>
|
||||
</span><span id="unnest_to_explode-154"><a href="#unnest_to_explode-154"><span class="linenos">154</span></a> <span class="n">this</span><span class="o">=</span><span class="n">udtf</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">e</span><span class="p">),</span>
|
||||
</span><span id="unnest_to_explode-155"><a href="#unnest_to_explode-155"><span class="linenos">155</span></a> <span class="n">view</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="unnest_to_explode-156"><a href="#unnest_to_explode-156"><span class="linenos">156</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">alias</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">column</span><span class="p">]),</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="unnest_to_explode-157"><a href="#unnest_to_explode-157"><span class="linenos">157</span></a> <span class="p">),</span>
|
||||
</span><span id="unnest_to_explode-158"><a href="#unnest_to_explode-158"><span class="linenos">158</span></a> <span class="p">)</span>
|
||||
</span><span id="unnest_to_explode-159"><a href="#unnest_to_explode-159"><span class="linenos">159</span></a>
|
||||
</span><span id="unnest_to_explode-160"><a href="#unnest_to_explode-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -680,65 +684,65 @@ other expressions. This transforms removes the precision from parameterized type
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#explode_to_unnest"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="explode_to_unnest-161"><a href="#explode_to_unnest-161"><span class="linenos">161</span></a><span class="k">def</span> <span class="nf">explode_to_unnest</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-162"><a href="#explode_to_unnest-162"><span class="linenos">162</span></a><span class="w"> </span><span class="sd">"""Convert explode/posexplode into unnest (used in hive -> presto)."""</span>
|
||||
</span><span id="explode_to_unnest-163"><a href="#explode_to_unnest-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-164"><a href="#explode_to_unnest-164"><span class="linenos">164</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span>
|
||||
</span><span id="explode_to_unnest-165"><a href="#explode_to_unnest-165"><span class="linenos">165</span></a>
|
||||
</span><span id="explode_to_unnest-166"><a href="#explode_to_unnest-166"><span class="linenos">166</span></a> <span class="n">taken_select_names</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-167"><a href="#explode_to_unnest-167"><span class="linenos">167</span></a> <span class="n">taken_source_names</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">Scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">references</span><span class="p">}</span>
|
||||
</span><span id="explode_to_unnest-168"><a href="#explode_to_unnest-168"><span class="linenos">168</span></a>
|
||||
</span><span id="explode_to_unnest-169"><a href="#explode_to_unnest-169"><span class="linenos">169</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-170"><a href="#explode_to_unnest-170"><span class="linenos">170</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="explode_to_unnest-171"><a href="#explode_to_unnest-171"><span class="linenos">171</span></a>
|
||||
</span><span id="explode_to_unnest-172"><a href="#explode_to_unnest-172"><span class="linenos">172</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="explode_to_unnest-173"><a href="#explode_to_unnest-173"><span class="linenos">173</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="explode_to_unnest-174"><a href="#explode_to_unnest-174"><span class="linenos">174</span></a>
|
||||
</span><span id="explode_to_unnest-175"><a href="#explode_to_unnest-175"><span class="linenos">175</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-176"><a href="#explode_to_unnest-176"><span class="linenos">176</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="explode_to_unnest-177"><a href="#explode_to_unnest-177"><span class="linenos">177</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="explode_to_unnest-178"><a href="#explode_to_unnest-178"><span class="linenos">178</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Aliases</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-179"><a href="#explode_to_unnest-179"><span class="linenos">179</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="explode_to_unnest-180"><a href="#explode_to_unnest-180"><span class="linenos">180</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="explode_to_unnest-181"><a href="#explode_to_unnest-181"><span class="linenos">181</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="explode_to_unnest-182"><a href="#explode_to_unnest-182"><span class="linenos">182</span></a>
|
||||
</span><span id="explode_to_unnest-183"><a href="#explode_to_unnest-183"><span class="linenos">183</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Explode</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)):</span>
|
||||
</span><span id="explode_to_unnest-184"><a href="#explode_to_unnest-184"><span class="linenos">184</span></a> <span class="n">is_posexplode</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-185"><a href="#explode_to_unnest-185"><span class="linenos">185</span></a>
|
||||
</span><span id="explode_to_unnest-186"><a href="#explode_to_unnest-186"><span class="linenos">186</span></a> <span class="n">explode_arg</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="explode_to_unnest-187"><a href="#explode_to_unnest-187"><span class="linenos">187</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">copy</span><span class="p">()],</span> <span class="n">ordinality</span><span class="o">=</span><span class="n">is_posexplode</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-188"><a href="#explode_to_unnest-188"><span class="linenos">188</span></a>
|
||||
</span><span id="explode_to_unnest-189"><a href="#explode_to_unnest-189"><span class="linenos">189</span></a> <span class="c1"># This ensures that we won't use [POS]EXPLODE's argument as a new selection</span>
|
||||
</span><span id="explode_to_unnest-190"><a href="#explode_to_unnest-190"><span class="linenos">190</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">explode_arg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-191"><a href="#explode_to_unnest-191"><span class="linenos">191</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">output_name</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-192"><a href="#explode_to_unnest-192"><span class="linenos">192</span></a>
|
||||
</span><span id="explode_to_unnest-193"><a href="#explode_to_unnest-193"><span class="linenos">193</span></a> <span class="n">unnest_source_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_source_names</span><span class="p">,</span> <span class="s2">"_u"</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-194"><a href="#explode_to_unnest-194"><span class="linenos">194</span></a> <span class="n">taken_source_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">unnest_source_alias</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-195"><a href="#explode_to_unnest-195"><span class="linenos">195</span></a>
|
||||
</span><span id="explode_to_unnest-196"><a href="#explode_to_unnest-196"><span class="linenos">196</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">explode_alias</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-197"><a href="#explode_to_unnest-197"><span class="linenos">197</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"col"</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-198"><a href="#explode_to_unnest-198"><span class="linenos">198</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-199"><a href="#explode_to_unnest-199"><span class="linenos">199</span></a>
|
||||
</span><span id="explode_to_unnest-200"><a href="#explode_to_unnest-200"><span class="linenos">200</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-201"><a href="#explode_to_unnest-201"><span class="linenos">201</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"pos"</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-202"><a href="#explode_to_unnest-202"><span class="linenos">202</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-203"><a href="#explode_to_unnest-203"><span class="linenos">203</span></a>
|
||||
</span><span id="explode_to_unnest-204"><a href="#explode_to_unnest-204"><span class="linenos">204</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-205"><a href="#explode_to_unnest-205"><span class="linenos">205</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">,</span> <span class="n">pos_alias</span><span class="p">]</span>
|
||||
</span><span id="explode_to_unnest-206"><a href="#explode_to_unnest-206"><span class="linenos">206</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="explode_to_unnest-207"><a href="#explode_to_unnest-207"><span class="linenos">207</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">,</span> <span class="n">explode_alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-208"><a href="#explode_to_unnest-208"><span class="linenos">208</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-209"><a href="#explode_to_unnest-209"><span class="linenos">209</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">]</span>
|
||||
</span><span id="explode_to_unnest-210"><a href="#explode_to_unnest-210"><span class="linenos">210</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">))</span>
|
||||
</span><span id="explode_to_unnest-211"><a href="#explode_to_unnest-211"><span class="linenos">211</span></a>
|
||||
</span><span id="explode_to_unnest-212"><a href="#explode_to_unnest-212"><span class="linenos">212</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">unnest_source_alias</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">column_names</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="explode_to_unnest-163"><a href="#explode_to_unnest-163"><span class="linenos">163</span></a><span class="k">def</span> <span class="nf">explode_to_unnest</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-164"><a href="#explode_to_unnest-164"><span class="linenos">164</span></a><span class="w"> </span><span class="sd">"""Convert explode/posexplode into unnest (used in hive -> presto)."""</span>
|
||||
</span><span id="explode_to_unnest-165"><a href="#explode_to_unnest-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-166"><a href="#explode_to_unnest-166"><span class="linenos">166</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span>
|
||||
</span><span id="explode_to_unnest-167"><a href="#explode_to_unnest-167"><span class="linenos">167</span></a>
|
||||
</span><span id="explode_to_unnest-168"><a href="#explode_to_unnest-168"><span class="linenos">168</span></a> <span class="n">taken_select_names</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-169"><a href="#explode_to_unnest-169"><span class="linenos">169</span></a> <span class="n">taken_source_names</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">Scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">references</span><span class="p">}</span>
|
||||
</span><span id="explode_to_unnest-170"><a href="#explode_to_unnest-170"><span class="linenos">170</span></a>
|
||||
</span><span id="explode_to_unnest-171"><a href="#explode_to_unnest-171"><span class="linenos">171</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-172"><a href="#explode_to_unnest-172"><span class="linenos">172</span></a> <span class="n">to_replace</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="explode_to_unnest-173"><a href="#explode_to_unnest-173"><span class="linenos">173</span></a>
|
||||
</span><span id="explode_to_unnest-174"><a href="#explode_to_unnest-174"><span class="linenos">174</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="explode_to_unnest-175"><a href="#explode_to_unnest-175"><span class="linenos">175</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="s2">""</span>
|
||||
</span><span id="explode_to_unnest-176"><a href="#explode_to_unnest-176"><span class="linenos">176</span></a>
|
||||
</span><span id="explode_to_unnest-177"><a href="#explode_to_unnest-177"><span class="linenos">177</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-178"><a href="#explode_to_unnest-178"><span class="linenos">178</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="explode_to_unnest-179"><a href="#explode_to_unnest-179"><span class="linenos">179</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="explode_to_unnest-180"><a href="#explode_to_unnest-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Aliases</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-181"><a href="#explode_to_unnest-181"><span class="linenos">181</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="explode_to_unnest-182"><a href="#explode_to_unnest-182"><span class="linenos">182</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">aliases</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="explode_to_unnest-183"><a href="#explode_to_unnest-183"><span class="linenos">183</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="explode_to_unnest-184"><a href="#explode_to_unnest-184"><span class="linenos">184</span></a>
|
||||
</span><span id="explode_to_unnest-185"><a href="#explode_to_unnest-185"><span class="linenos">185</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Explode</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)):</span>
|
||||
</span><span id="explode_to_unnest-186"><a href="#explode_to_unnest-186"><span class="linenos">186</span></a> <span class="n">is_posexplode</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Posexplode</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-187"><a href="#explode_to_unnest-187"><span class="linenos">187</span></a>
|
||||
</span><span id="explode_to_unnest-188"><a href="#explode_to_unnest-188"><span class="linenos">188</span></a> <span class="n">explode_arg</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="explode_to_unnest-189"><a href="#explode_to_unnest-189"><span class="linenos">189</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">(</span><span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">copy</span><span class="p">()],</span> <span class="n">ordinality</span><span class="o">=</span><span class="n">is_posexplode</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-190"><a href="#explode_to_unnest-190"><span class="linenos">190</span></a>
|
||||
</span><span id="explode_to_unnest-191"><a href="#explode_to_unnest-191"><span class="linenos">191</span></a> <span class="c1"># This ensures that we won't use [POS]EXPLODE's argument as a new selection</span>
|
||||
</span><span id="explode_to_unnest-192"><a href="#explode_to_unnest-192"><span class="linenos">192</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">explode_arg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-193"><a href="#explode_to_unnest-193"><span class="linenos">193</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_arg</span><span class="o">.</span><span class="n">output_name</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-194"><a href="#explode_to_unnest-194"><span class="linenos">194</span></a>
|
||||
</span><span id="explode_to_unnest-195"><a href="#explode_to_unnest-195"><span class="linenos">195</span></a> <span class="n">unnest_source_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_source_names</span><span class="p">,</span> <span class="s2">"_u"</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-196"><a href="#explode_to_unnest-196"><span class="linenos">196</span></a> <span class="n">taken_source_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">unnest_source_alias</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-197"><a href="#explode_to_unnest-197"><span class="linenos">197</span></a>
|
||||
</span><span id="explode_to_unnest-198"><a href="#explode_to_unnest-198"><span class="linenos">198</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">explode_alias</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-199"><a href="#explode_to_unnest-199"><span class="linenos">199</span></a> <span class="n">explode_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"col"</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-200"><a href="#explode_to_unnest-200"><span class="linenos">200</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-201"><a href="#explode_to_unnest-201"><span class="linenos">201</span></a>
|
||||
</span><span id="explode_to_unnest-202"><a href="#explode_to_unnest-202"><span class="linenos">202</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-203"><a href="#explode_to_unnest-203"><span class="linenos">203</span></a> <span class="n">pos_alias</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken_select_names</span><span class="p">,</span> <span class="s2">"pos"</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-204"><a href="#explode_to_unnest-204"><span class="linenos">204</span></a> <span class="n">taken_select_names</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-205"><a href="#explode_to_unnest-205"><span class="linenos">205</span></a>
|
||||
</span><span id="explode_to_unnest-206"><a href="#explode_to_unnest-206"><span class="linenos">206</span></a> <span class="k">if</span> <span class="n">is_posexplode</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-207"><a href="#explode_to_unnest-207"><span class="linenos">207</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">,</span> <span class="n">pos_alias</span><span class="p">]</span>
|
||||
</span><span id="explode_to_unnest-208"><a href="#explode_to_unnest-208"><span class="linenos">208</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="explode_to_unnest-209"><a href="#explode_to_unnest-209"><span class="linenos">209</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">pos_alias</span><span class="p">,</span> <span class="n">explode_alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-210"><a href="#explode_to_unnest-210"><span class="linenos">210</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-211"><a href="#explode_to_unnest-211"><span class="linenos">211</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">explode_alias</span><span class="p">]</span>
|
||||
</span><span id="explode_to_unnest-212"><a href="#explode_to_unnest-212"><span class="linenos">212</span></a> <span class="n">to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">explode_alias</span><span class="p">))</span>
|
||||
</span><span id="explode_to_unnest-213"><a href="#explode_to_unnest-213"><span class="linenos">213</span></a>
|
||||
</span><span id="explode_to_unnest-214"><a href="#explode_to_unnest-214"><span class="linenos">214</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-215"><a href="#explode_to_unnest-215"><span class="linenos">215</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-216"><a href="#explode_to_unnest-216"><span class="linenos">216</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-217"><a href="#explode_to_unnest-217"><span class="linenos">217</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">join_type</span><span class="o">=</span><span class="s2">"CROSS"</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-218"><a href="#explode_to_unnest-218"><span class="linenos">218</span></a>
|
||||
</span><span id="explode_to_unnest-219"><a href="#explode_to_unnest-219"><span class="linenos">219</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="explode_to_unnest-214"><a href="#explode_to_unnest-214"><span class="linenos">214</span></a> <span class="n">unnest</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">unnest_source_alias</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">column_names</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-215"><a href="#explode_to_unnest-215"><span class="linenos">215</span></a>
|
||||
</span><span id="explode_to_unnest-216"><a href="#explode_to_unnest-216"><span class="linenos">216</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">):</span>
|
||||
</span><span id="explode_to_unnest-217"><a href="#explode_to_unnest-217"><span class="linenos">217</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-218"><a href="#explode_to_unnest-218"><span class="linenos">218</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="explode_to_unnest-219"><a href="#explode_to_unnest-219"><span class="linenos">219</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">unnest</span><span class="p">,</span> <span class="n">join_type</span><span class="o">=</span><span class="s2">"CROSS"</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="explode_to_unnest-220"><a href="#explode_to_unnest-220"><span class="linenos">220</span></a>
|
||||
</span><span id="explode_to_unnest-221"><a href="#explode_to_unnest-221"><span class="linenos">221</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -758,17 +762,17 @@ other expressions. This transforms removes the precision from parameterized type
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#remove_within_group_for_percentiles"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_within_group_for_percentiles-222"><a href="#remove_within_group_for_percentiles-222"><span class="linenos">222</span></a><span class="k">def</span> <span class="nf">remove_within_group_for_percentiles</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_within_group_for_percentiles-223"><a href="#remove_within_group_for_percentiles-223"><span class="linenos">223</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="remove_within_group_for_percentiles-224"><a href="#remove_within_group_for_percentiles-224"><span class="linenos">224</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">)</span>
|
||||
</span><span id="remove_within_group_for_percentiles-225"><a href="#remove_within_group_for_percentiles-225"><span class="linenos">225</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">PercentileCont</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">PercentileDisc</span><span class="p">))</span>
|
||||
</span><span id="remove_within_group_for_percentiles-226"><a href="#remove_within_group_for_percentiles-226"><span class="linenos">226</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
|
||||
</span><span id="remove_within_group_for_percentiles-227"><a href="#remove_within_group_for_percentiles-227"><span class="linenos">227</span></a> <span class="p">):</span>
|
||||
</span><span id="remove_within_group_for_percentiles-228"><a href="#remove_within_group_for_percentiles-228"><span class="linenos">228</span></a> <span class="n">quantile</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_within_group_for_percentiles-229"><a href="#remove_within_group_for_percentiles-229"><span class="linenos">229</span></a> <span class="n">input_value</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">))</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_within_group_for_percentiles-230"><a href="#remove_within_group_for_percentiles-230"><span class="linenos">230</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">ApproxQuantile</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">input_value</span><span class="p">,</span> <span class="n">quantile</span><span class="o">=</span><span class="n">quantile</span><span class="p">))</span>
|
||||
</span><span id="remove_within_group_for_percentiles-231"><a href="#remove_within_group_for_percentiles-231"><span class="linenos">231</span></a>
|
||||
</span><span id="remove_within_group_for_percentiles-232"><a href="#remove_within_group_for_percentiles-232"><span class="linenos">232</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_within_group_for_percentiles-224"><a href="#remove_within_group_for_percentiles-224"><span class="linenos">224</span></a><span class="k">def</span> <span class="nf">remove_within_group_for_percentiles</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="remove_within_group_for_percentiles-225"><a href="#remove_within_group_for_percentiles-225"><span class="linenos">225</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="remove_within_group_for_percentiles-226"><a href="#remove_within_group_for_percentiles-226"><span class="linenos">226</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">)</span>
|
||||
</span><span id="remove_within_group_for_percentiles-227"><a href="#remove_within_group_for_percentiles-227"><span class="linenos">227</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">PercentileCont</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">PercentileDisc</span><span class="p">))</span>
|
||||
</span><span id="remove_within_group_for_percentiles-228"><a href="#remove_within_group_for_percentiles-228"><span class="linenos">228</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
|
||||
</span><span id="remove_within_group_for_percentiles-229"><a href="#remove_within_group_for_percentiles-229"><span class="linenos">229</span></a> <span class="p">):</span>
|
||||
</span><span id="remove_within_group_for_percentiles-230"><a href="#remove_within_group_for_percentiles-230"><span class="linenos">230</span></a> <span class="n">quantile</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_within_group_for_percentiles-231"><a href="#remove_within_group_for_percentiles-231"><span class="linenos">231</span></a> <span class="n">input_value</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">))</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="remove_within_group_for_percentiles-232"><a href="#remove_within_group_for_percentiles-232"><span class="linenos">232</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">ApproxQuantile</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">input_value</span><span class="p">,</span> <span class="n">quantile</span><span class="o">=</span><span class="n">quantile</span><span class="p">))</span>
|
||||
</span><span id="remove_within_group_for_percentiles-233"><a href="#remove_within_group_for_percentiles-233"><span class="linenos">233</span></a>
|
||||
</span><span id="remove_within_group_for_percentiles-234"><a href="#remove_within_group_for_percentiles-234"><span class="linenos">234</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -786,22 +790,22 @@ other expressions. This transforms removes the precision from parameterized type
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#add_recursive_cte_column_names"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="add_recursive_cte_column_names-235"><a href="#add_recursive_cte_column_names-235"><span class="linenos">235</span></a><span class="k">def</span> <span class="nf">add_recursive_cte_column_names</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-236"><a href="#add_recursive_cte_column_names-236"><span class="linenos">236</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-237"><a href="#add_recursive_cte_column_names-237"><span class="linenos">237</span></a> <span class="n">next_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_c_"</span><span class="p">)</span>
|
||||
</span><span id="add_recursive_cte_column_names-238"><a href="#add_recursive_cte_column_names-238"><span class="linenos">238</span></a>
|
||||
</span><span id="add_recursive_cte_column_names-239"><a href="#add_recursive_cte_column_names-239"><span class="linenos">239</span></a> <span class="k">for</span> <span class="n">cte</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-240"><a href="#add_recursive_cte_column_names-240"><span class="linenos">240</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-241"><a href="#add_recursive_cte_column_names-241"><span class="linenos">241</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="add_recursive_cte_column_names-242"><a href="#add_recursive_cte_column_names-242"><span class="linenos">242</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="add_recursive_cte_column_names-243"><a href="#add_recursive_cte_column_names-243"><span class="linenos">243</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">query</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="add_recursive_cte_column_names-244"><a href="#add_recursive_cte_column_names-244"><span class="linenos">244</span></a>
|
||||
</span><span id="add_recursive_cte_column_names-245"><a href="#add_recursive_cte_column_names-245"><span class="linenos">245</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="add_recursive_cte_column_names-246"><a href="#add_recursive_cte_column_names-246"><span class="linenos">246</span></a> <span class="s2">"columns"</span><span class="p">,</span>
|
||||
</span><span id="add_recursive_cte_column_names-247"><a href="#add_recursive_cte_column_names-247"><span class="linenos">247</span></a> <span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">or</span> <span class="n">next_name</span><span class="p">())</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">query</span><span class="o">.</span><span class="n">selects</span><span class="p">],</span>
|
||||
</span><span id="add_recursive_cte_column_names-248"><a href="#add_recursive_cte_column_names-248"><span class="linenos">248</span></a> <span class="p">)</span>
|
||||
</span><span id="add_recursive_cte_column_names-249"><a href="#add_recursive_cte_column_names-249"><span class="linenos">249</span></a>
|
||||
</span><span id="add_recursive_cte_column_names-250"><a href="#add_recursive_cte_column_names-250"><span class="linenos">250</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="add_recursive_cte_column_names-237"><a href="#add_recursive_cte_column_names-237"><span class="linenos">237</span></a><span class="k">def</span> <span class="nf">add_recursive_cte_column_names</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-238"><a href="#add_recursive_cte_column_names-238"><span class="linenos">238</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">With</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-239"><a href="#add_recursive_cte_column_names-239"><span class="linenos">239</span></a> <span class="n">next_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_c_"</span><span class="p">)</span>
|
||||
</span><span id="add_recursive_cte_column_names-240"><a href="#add_recursive_cte_column_names-240"><span class="linenos">240</span></a>
|
||||
</span><span id="add_recursive_cte_column_names-241"><a href="#add_recursive_cte_column_names-241"><span class="linenos">241</span></a> <span class="k">for</span> <span class="n">cte</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-242"><a href="#add_recursive_cte_column_names-242"><span class="linenos">242</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="add_recursive_cte_column_names-243"><a href="#add_recursive_cte_column_names-243"><span class="linenos">243</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="add_recursive_cte_column_names-244"><a href="#add_recursive_cte_column_names-244"><span class="linenos">244</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="add_recursive_cte_column_names-245"><a href="#add_recursive_cte_column_names-245"><span class="linenos">245</span></a> <span class="n">query</span> <span class="o">=</span> <span class="n">query</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="add_recursive_cte_column_names-246"><a href="#add_recursive_cte_column_names-246"><span class="linenos">246</span></a>
|
||||
</span><span id="add_recursive_cte_column_names-247"><a href="#add_recursive_cte_column_names-247"><span class="linenos">247</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"alias"</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="add_recursive_cte_column_names-248"><a href="#add_recursive_cte_column_names-248"><span class="linenos">248</span></a> <span class="s2">"columns"</span><span class="p">,</span>
|
||||
</span><span id="add_recursive_cte_column_names-249"><a href="#add_recursive_cte_column_names-249"><span class="linenos">249</span></a> <span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">or</span> <span class="n">next_name</span><span class="p">())</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">query</span><span class="o">.</span><span class="n">selects</span><span class="p">],</span>
|
||||
</span><span id="add_recursive_cte_column_names-250"><a href="#add_recursive_cte_column_names-250"><span class="linenos">250</span></a> <span class="p">)</span>
|
||||
</span><span id="add_recursive_cte_column_names-251"><a href="#add_recursive_cte_column_names-251"><span class="linenos">251</span></a>
|
||||
</span><span id="add_recursive_cte_column_names-252"><a href="#add_recursive_cte_column_names-252"><span class="linenos">252</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -819,15 +823,15 @@ other expressions. This transforms removes the precision from parameterized type
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#epoch_cast_to_ts"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="epoch_cast_to_ts-253"><a href="#epoch_cast_to_ts-253"><span class="linenos">253</span></a><span class="k">def</span> <span class="nf">epoch_cast_to_ts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="epoch_cast_to_ts-254"><a href="#epoch_cast_to_ts-254"><span class="linenos">254</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="epoch_cast_to_ts-255"><a href="#epoch_cast_to_ts-255"><span class="linenos">255</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TryCast</span><span class="p">))</span>
|
||||
</span><span id="epoch_cast_to_ts-256"><a href="#epoch_cast_to_ts-256"><span class="linenos">256</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"epoch"</span>
|
||||
</span><span id="epoch_cast_to_ts-257"><a href="#epoch_cast_to_ts-257"><span class="linenos">257</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="epoch_cast_to_ts-258"><a href="#epoch_cast_to_ts-258"><span class="linenos">258</span></a> <span class="p">):</span>
|
||||
</span><span id="epoch_cast_to_ts-259"><a href="#epoch_cast_to_ts-259"><span class="linenos">259</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="o">.</span><span class="n">string</span><span class="p">(</span><span class="s2">"1970-01-01 00:00:00"</span><span class="p">))</span>
|
||||
</span><span id="epoch_cast_to_ts-260"><a href="#epoch_cast_to_ts-260"><span class="linenos">260</span></a>
|
||||
</span><span id="epoch_cast_to_ts-261"><a href="#epoch_cast_to_ts-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="epoch_cast_to_ts-255"><a href="#epoch_cast_to_ts-255"><span class="linenos">255</span></a><span class="k">def</span> <span class="nf">epoch_cast_to_ts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="epoch_cast_to_ts-256"><a href="#epoch_cast_to_ts-256"><span class="linenos">256</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="epoch_cast_to_ts-257"><a href="#epoch_cast_to_ts-257"><span class="linenos">257</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TryCast</span><span class="p">))</span>
|
||||
</span><span id="epoch_cast_to_ts-258"><a href="#epoch_cast_to_ts-258"><span class="linenos">258</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s2">"epoch"</span>
|
||||
</span><span id="epoch_cast_to_ts-259"><a href="#epoch_cast_to_ts-259"><span class="linenos">259</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
|
||||
</span><span id="epoch_cast_to_ts-260"><a href="#epoch_cast_to_ts-260"><span class="linenos">260</span></a> <span class="p">):</span>
|
||||
</span><span id="epoch_cast_to_ts-261"><a href="#epoch_cast_to_ts-261"><span class="linenos">261</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="o">.</span><span class="n">string</span><span class="p">(</span><span class="s2">"1970-01-01 00:00:00"</span><span class="p">))</span>
|
||||
</span><span id="epoch_cast_to_ts-262"><a href="#epoch_cast_to_ts-262"><span class="linenos">262</span></a>
|
||||
</span><span id="epoch_cast_to_ts-263"><a href="#epoch_cast_to_ts-263"><span class="linenos">263</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -845,47 +849,47 @@ other expressions. This transforms removes the precision from parameterized type
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#preprocess"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="preprocess-264"><a href="#preprocess-264"><span class="linenos">264</span></a><span class="k">def</span> <span class="nf">preprocess</span><span class="p">(</span>
|
||||
</span><span id="preprocess-265"><a href="#preprocess-265"><span class="linenos">265</span></a> <span class="n">transforms</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]],</span>
|
||||
</span><span id="preprocess-266"><a href="#preprocess-266"><span class="linenos">266</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">Generator</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="preprocess-267"><a href="#preprocess-267"><span class="linenos">267</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="preprocess-268"><a href="#preprocess-268"><span class="linenos">268</span></a><span class="sd"> Creates a new transform by chaining a sequence of transformations and converts the resulting</span>
|
||||
</span><span id="preprocess-269"><a href="#preprocess-269"><span class="linenos">269</span></a><span class="sd"> expression to SQL, using either the "_sql" method corresponding to the resulting expression,</span>
|
||||
</span><span id="preprocess-270"><a href="#preprocess-270"><span class="linenos">270</span></a><span class="sd"> or the appropriate `Generator.TRANSFORMS` function (when applicable -- see below).</span>
|
||||
</span><span id="preprocess-271"><a href="#preprocess-271"><span class="linenos">271</span></a>
|
||||
</span><span id="preprocess-272"><a href="#preprocess-272"><span class="linenos">272</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="preprocess-273"><a href="#preprocess-273"><span class="linenos">273</span></a><span class="sd"> transforms: sequence of transform functions. These will be called in order.</span>
|
||||
</span><span id="preprocess-274"><a href="#preprocess-274"><span class="linenos">274</span></a>
|
||||
</span><span id="preprocess-275"><a href="#preprocess-275"><span class="linenos">275</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="preprocess-276"><a href="#preprocess-276"><span class="linenos">276</span></a><span class="sd"> Function that can be used as a generator transform.</span>
|
||||
</span><span id="preprocess-277"><a href="#preprocess-277"><span class="linenos">277</span></a><span class="sd"> """</span>
|
||||
</span><span id="preprocess-278"><a href="#preprocess-278"><span class="linenos">278</span></a>
|
||||
</span><span id="preprocess-279"><a href="#preprocess-279"><span class="linenos">279</span></a> <span class="k">def</span> <span class="nf">_to_sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="preprocess-280"><a href="#preprocess-280"><span class="linenos">280</span></a> <span class="n">expression_type</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-281"><a href="#preprocess-281"><span class="linenos">281</span></a>
|
||||
</span><span id="preprocess-282"><a href="#preprocess-282"><span class="linenos">282</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">0</span><span class="p">](</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="preprocess-283"><a href="#preprocess-283"><span class="linenos">283</span></a> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="preprocess-284"><a href="#preprocess-284"><span class="linenos">284</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">t</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-285"><a href="#preprocess-285"><span class="linenos">285</span></a>
|
||||
</span><span id="preprocess-286"><a href="#preprocess-286"><span class="linenos">286</span></a> <span class="n">_sql_handler</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">key</span> <span class="o">+</span> <span class="s2">"_sql"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="preprocess-287"><a href="#preprocess-287"><span class="linenos">287</span></a> <span class="k">if</span> <span class="n">_sql_handler</span><span class="p">:</span>
|
||||
</span><span id="preprocess-288"><a href="#preprocess-288"><span class="linenos">288</span></a> <span class="k">return</span> <span class="n">_sql_handler</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-289"><a href="#preprocess-289"><span class="linenos">289</span></a>
|
||||
</span><span id="preprocess-290"><a href="#preprocess-290"><span class="linenos">290</span></a> <span class="n">transforms_handler</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">TRANSFORMS</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">))</span>
|
||||
</span><span id="preprocess-291"><a href="#preprocess-291"><span class="linenos">291</span></a> <span class="k">if</span> <span class="n">transforms_handler</span><span class="p">:</span>
|
||||
</span><span id="preprocess-292"><a href="#preprocess-292"><span class="linenos">292</span></a> <span class="c1"># Ensures we don't enter an infinite loop. This can happen when the original expression</span>
|
||||
</span><span id="preprocess-293"><a href="#preprocess-293"><span class="linenos">293</span></a> <span class="c1"># has the same type as the final expression and there's no _sql method available for it,</span>
|
||||
</span><span id="preprocess-294"><a href="#preprocess-294"><span class="linenos">294</span></a> <span class="c1"># because then it'd re-enter _to_sql.</span>
|
||||
</span><span id="preprocess-295"><a href="#preprocess-295"><span class="linenos">295</span></a> <span class="k">if</span> <span class="n">expression_type</span> <span class="ow">is</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="preprocess-296"><a href="#preprocess-296"><span class="linenos">296</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
||||
</span><span id="preprocess-297"><a href="#preprocess-297"><span class="linenos">297</span></a> <span class="sa">f</span><span class="s2">"Expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2"> requires a _sql method in order to be transformed."</span>
|
||||
</span><span id="preprocess-298"><a href="#preprocess-298"><span class="linenos">298</span></a> <span class="p">)</span>
|
||||
</span><span id="preprocess-299"><a href="#preprocess-299"><span class="linenos">299</span></a>
|
||||
</span><span id="preprocess-300"><a href="#preprocess-300"><span class="linenos">300</span></a> <span class="k">return</span> <span class="n">transforms_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="preprocess-266"><a href="#preprocess-266"><span class="linenos">266</span></a><span class="k">def</span> <span class="nf">preprocess</span><span class="p">(</span>
|
||||
</span><span id="preprocess-267"><a href="#preprocess-267"><span class="linenos">267</span></a> <span class="n">transforms</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]],</span>
|
||||
</span><span id="preprocess-268"><a href="#preprocess-268"><span class="linenos">268</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">Generator</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="preprocess-269"><a href="#preprocess-269"><span class="linenos">269</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="preprocess-270"><a href="#preprocess-270"><span class="linenos">270</span></a><span class="sd"> Creates a new transform by chaining a sequence of transformations and converts the resulting</span>
|
||||
</span><span id="preprocess-271"><a href="#preprocess-271"><span class="linenos">271</span></a><span class="sd"> expression to SQL, using either the "_sql" method corresponding to the resulting expression,</span>
|
||||
</span><span id="preprocess-272"><a href="#preprocess-272"><span class="linenos">272</span></a><span class="sd"> or the appropriate `Generator.TRANSFORMS` function (when applicable -- see below).</span>
|
||||
</span><span id="preprocess-273"><a href="#preprocess-273"><span class="linenos">273</span></a>
|
||||
</span><span id="preprocess-274"><a href="#preprocess-274"><span class="linenos">274</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="preprocess-275"><a href="#preprocess-275"><span class="linenos">275</span></a><span class="sd"> transforms: sequence of transform functions. These will be called in order.</span>
|
||||
</span><span id="preprocess-276"><a href="#preprocess-276"><span class="linenos">276</span></a>
|
||||
</span><span id="preprocess-277"><a href="#preprocess-277"><span class="linenos">277</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="preprocess-278"><a href="#preprocess-278"><span class="linenos">278</span></a><span class="sd"> Function that can be used as a generator transform.</span>
|
||||
</span><span id="preprocess-279"><a href="#preprocess-279"><span class="linenos">279</span></a><span class="sd"> """</span>
|
||||
</span><span id="preprocess-280"><a href="#preprocess-280"><span class="linenos">280</span></a>
|
||||
</span><span id="preprocess-281"><a href="#preprocess-281"><span class="linenos">281</span></a> <span class="k">def</span> <span class="nf">_to_sql</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="preprocess-282"><a href="#preprocess-282"><span class="linenos">282</span></a> <span class="n">expression_type</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-283"><a href="#preprocess-283"><span class="linenos">283</span></a>
|
||||
</span><span id="preprocess-284"><a href="#preprocess-284"><span class="linenos">284</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">0</span><span class="p">](</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="preprocess-285"><a href="#preprocess-285"><span class="linenos">285</span></a> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">transforms</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="preprocess-286"><a href="#preprocess-286"><span class="linenos">286</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">t</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-287"><a href="#preprocess-287"><span class="linenos">287</span></a>
|
||||
</span><span id="preprocess-288"><a href="#preprocess-288"><span class="linenos">288</span></a> <span class="n">_sql_handler</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">key</span> <span class="o">+</span> <span class="s2">"_sql"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="preprocess-289"><a href="#preprocess-289"><span class="linenos">289</span></a> <span class="k">if</span> <span class="n">_sql_handler</span><span class="p">:</span>
|
||||
</span><span id="preprocess-290"><a href="#preprocess-290"><span class="linenos">290</span></a> <span class="k">return</span> <span class="n">_sql_handler</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-291"><a href="#preprocess-291"><span class="linenos">291</span></a>
|
||||
</span><span id="preprocess-292"><a href="#preprocess-292"><span class="linenos">292</span></a> <span class="n">transforms_handler</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">TRANSFORMS</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">))</span>
|
||||
</span><span id="preprocess-293"><a href="#preprocess-293"><span class="linenos">293</span></a> <span class="k">if</span> <span class="n">transforms_handler</span><span class="p">:</span>
|
||||
</span><span id="preprocess-294"><a href="#preprocess-294"><span class="linenos">294</span></a> <span class="c1"># Ensures we don't enter an infinite loop. This can happen when the original expression</span>
|
||||
</span><span id="preprocess-295"><a href="#preprocess-295"><span class="linenos">295</span></a> <span class="c1"># has the same type as the final expression and there's no _sql method available for it,</span>
|
||||
</span><span id="preprocess-296"><a href="#preprocess-296"><span class="linenos">296</span></a> <span class="c1"># because then it'd re-enter _to_sql.</span>
|
||||
</span><span id="preprocess-297"><a href="#preprocess-297"><span class="linenos">297</span></a> <span class="k">if</span> <span class="n">expression_type</span> <span class="ow">is</span> <span class="nb">type</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="preprocess-298"><a href="#preprocess-298"><span class="linenos">298</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
|
||||
</span><span id="preprocess-299"><a href="#preprocess-299"><span class="linenos">299</span></a> <span class="sa">f</span><span class="s2">"Expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2"> requires a _sql method in order to be transformed."</span>
|
||||
</span><span id="preprocess-300"><a href="#preprocess-300"><span class="linenos">300</span></a> <span class="p">)</span>
|
||||
</span><span id="preprocess-301"><a href="#preprocess-301"><span class="linenos">301</span></a>
|
||||
</span><span id="preprocess-302"><a href="#preprocess-302"><span class="linenos">302</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unsupported expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="preprocess-302"><a href="#preprocess-302"><span class="linenos">302</span></a> <span class="k">return</span> <span class="n">transforms_handler</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="preprocess-303"><a href="#preprocess-303"><span class="linenos">303</span></a>
|
||||
</span><span id="preprocess-304"><a href="#preprocess-304"><span class="linenos">304</span></a> <span class="k">return</span> <span class="n">_to_sql</span>
|
||||
</span><span id="preprocess-304"><a href="#preprocess-304"><span class="linenos">304</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unsupported expression type </span><span class="si">{</span><span class="n">expression</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="preprocess-305"><a href="#preprocess-305"><span class="linenos">305</span></a>
|
||||
</span><span id="preprocess-306"><a href="#preprocess-306"><span class="linenos">306</span></a> <span class="k">return</span> <span class="n">_to_sql</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue