ɯeivdZddlZddlZddlZddlZddlmZddlmZm Z ddl m Z ddl mZddlmZddlmZmZmZmZmZmZGdd eed ZGd d eeZee _GddeeZee _GddeeZGddeeZee _ee _GddeeZ GddeeZ!GddeeZ"GddeeZ#GddeeZ$Gdd eeZ%Gd!d"eeZ&Gd#d$eeZ'Gd%d&eeZ(Gd'd(eeZ)Gd)d*eeZ*Gd+d,eeZ+Gd-d.eeZ,Gd/d0eeZ-Gd1d2eeZ.Gd3d4eeZ/Gd5d6eeZ0Gd7d8eeZ1Gd9d:eeZ2Gd;deeZ4Gd?d@eeZ5GdAdBeeZ6GdCdDeeZ7GdEdFee8Z9GdGdHeeZ:GdIdJeeZ;GdKdLeeZ<GdMdNeeZ=GdOdPe eZ>GdQdRe eZ?ee?e>e?_@GdSdTe eZAGdUdVe eZBeeBeAeB_@GdWdXeeZCGdYdZeeZDGd[d\eeZEGd]d^eeZFGd_d`eeZGGdadbeeZHGdcddeeZIGdedfeeZJdhdgZKeKy)izBModule houses Modin configs originated from environment variables.N)dedent)AnyOptional)version)doc) _TYPE_PARAMS_UNSETDeprecationDescriptorExactStr Parameter ValueSourcecReZdZUdZdZeeed<edefdZ edefdZ y)EnvironmentVariablez9Base class for environment variables-based configuration.Nvarnamereturncj|j tdtj|jS)a  Read the value from environment variable. Returns ------- str Config raw value. Raises ------ TypeError If `varname` is None. KeyError If value is absent. zvarname should not be None)r TypeErrorosenvironclss i/var/www/html/glpi_dashboard/venv/lib/python3.12/site-packages/snowflake/snowpark/modin/config/envvars.py_get_raw_from_configz(EnvironmentVariable._get_raw_from_config-s-" ;; 89 9zz#++&&c|jdt|jxsdjdt|j j }|jr)|ddjd|jDdz }|S)zm Generate user-presentable help for the config. Returns ------- str z: Unknownz Provide z (valid examples are: , c32K|]}t|yw)N)str).0cs r z/EnvironmentVariable.get_help..Ms6S!s1v6Ss)) rr__doc__striprtypehelpchoicesjoin)rr's rget_helpzEnvironmentVariable.get_helpBs++b (@y!A!G!G!I J,WcdgdldlWmWrWrVst ;; ,TYY6Ss{{6S-S,TTUV VD r) __name__ __module__ __qualname__r$rrr__annotations__ classmethodrr*rrrr(sGC!GXc]!'S''(   rrT)r&abstractceZdZdZdZdZy)$SnowflakeModinTelemetryFlushIntervalz Minimum number of seconds between a flush of telemetry to snowflake from metrics generated in the client modin layer. (SNOWFLAKE_MODIN_TELEMETRY_FLUSH_INTERVALNr+r,r-r$rdefaultr0rrr3r3Qs 9GGrr3)r&ceZdZdZdZdZy)SnowflakeModinTelemetryEnabledz Enable or disable telemetry sent to Snowflake from the modin client. This only includes telemetry sent through the modin metrics events, not all snowpark telemetry generated through lazily evaluated queries on the Snowflake backend. !SNOWFLAKE_MODIN_TELEMETRY_ENABLEDTNr6r0rrr9r9^s2GGrr9ceZdZdZdZdZy) SnowflakePandasTransferThresholdz{ Targeted max number of dataframe rows which should be transferred from Snowflake when using hybrid execution. SNOWFLAKE_PANDAS_MAX_XFER_ROWSiNr6r0rrr<r<ms /GGrr<ceZdZdZdZdZy)&PandasToSnowflakeParquetThresholdBytesz When a pandas-backend dataframe's shallow memory usage exceeds this threshold, implement to_snowflake() by writing the dataframe to a parquet file and loading the parquet file into Snowflake. .SNOWFLAKE_PANDAS_MAX_TO_SNOWFLAKE_MEMORY_BYTESi-Nr6r0rrr?r?ws ?GGrr?cpeZdZdZdZededfdZedeffd Z ededdffd Z xZ S) EnvWithSibilingsz9Ensure values synchronization between sibling parameters.Trct)zReturn a sibling parameter.)NotImplementedErrorrs r_siblingzEnvWithSibilings._siblings "##rc|j}|jtur|jturt|t j 5t jdttt|ddd|j|jcxk(rtjk(r)nn&td|jd|jdddlm}|jtjk(rT|j#|jtj$k7 |j|_tj|_ n|jtjk(rT|j#|jtj$k7 |j|_tj|_ n]|j#|jtj$k7xs|jtj$k7 |j|_t|S#1swYxYw) z Get parameter's value and ensure that it's equal to the sibling's value. Returns ------- Any ignorecategoryNz,Configuration is ambiguous. You cannot set 'z' and 'z' at the same time.r) ErrorMessage)failure_condition)rE_valuer supergetwarningscatch_warningsfilterwarnings FutureWarningrB _value_sourcer GOT_FROM_CFG_SOURCE ValueErrorr3snowflake.snowpark.modin.plugin.utils.error_messagerJcatch_bugs_and_request_emailDEFAULT)rsiblingrJ __class__s rrNzEnvWithSibilings.gets,,. >>V # f(< GKM((* 7''=I&46 7 !!((3223!B3;;-wW^WfWfVggz{ Y  K$C$CC99&-&;&;{?R?R&R:"%(3(G(G%&&+*I*II99&)&7&7;;N;N&N:%^^ $/$C$C!99&)&7&7;;N;N&N'D,, 0C0CC: "%w{}M 7 7s .H99IvalueNc:t|||jrgd|_ tj5tj dt |jj|dddd|_yy#1swYxYw#d|_wxYw)z Set a new value to this parameter as well as to its sibling. Parameters ---------- value : Any FrGrHNT)rMput_update_siblingrOrPrQrRrErr[rZs rr]zEnvWithSibilings.puts  E   "'C  +,,..++H}MLLN&&u-. '+# .. '+#s"B;B4BB B B) r+r,r-r$r^r/r&rErrNr] __classcell__rZs@rrBrBsk DO$01$$2C22h++++rrBceZdZdZdZy)IsDebugzDForce Modin engine to be "Python" unless specified by $MODIN_ENGINE. MODIN_DEBUGNr+r,r-r$rr0rrrcrcs NGrrcceZdZdZdZdZdhZdZede fdZ ee e jjdedeffd Z xZS) Enginez&Distribution engine to run queries by. MODIN_ENGINE)RayDaskPythonNativeUnidistrkFrcddlm}m}m}tj s |j ry ddl}tj|j|krtdd|dzy#t$rYnwxYw ddl }ddl }tj|j|ks"tj|j|krtd |dy #t$rYnwxYw dd lm}y #t$rYnwxYw ddl}tj|j|krtd d|dzy#t$r YtdwxYw)z_ Get default value of the config. Returns ------- str r)MIN_DASK_VERSIONMIN_RAY_VERSIONMIN_UNIDIST_VERSIONrkNzDPlease `pip install "modin[ray]"` to install compatible Ray version z(>=z).rizIPlease `pip install "modin[dask]"` to install compatible Dask version (>=rj)DbWorkerrlzQPlease `pip install "unidist[mpi]"` to install compatible unidist on MPI version rmzDPlease refer to installation documentation page to install an engine)snowflake.snowpark.modin.utilsrorprqrcrNhas_custom_enginerayrparse __version__ ImportErrordask distributedPmodin.experimental.core.execution.native.implementations.hdk_on_native.db_workerrrunidist) rrorprqruryrzrrr|s r _get_defaultzEngine._get_defaults   ;;=C11   }}S__-?!!O,B/0         d../2BB==!8!89 !<<> !r) r+r,r-r$rr/rrr}r0rrrrsEM!G" "S " "rrceZdZdZdZy)HdkFragmentSizezDHow big a fragment in HDK should be when creating a table (in rows).MODIN_HDK_FRAGMENT_SIZENrer0rrrrs N'GrrceZdZdZdZdZy) DoUseCalcitez1Whether to use Calcite for HDK queries execution.MODIN_USE_CALCITETNr6r0rrrrs;!GGrrceZdZdZdZdZy)TestDatasetSizez$Dataset size for running some tests.MODIN_TEST_DATASET_SIZE)SmallNormalBigN)r+r,r-r$rr(r0rrrrs.'G(Grrc2eZdZdZdZej dk7Zy)TrackFileLeaksz>Whether to track for open file handles leakage during testing.MODIN_TEST_TRACK_FILE_LEAKSwin32N)r+r,r-r$rsysplatformr7r0rrrrsH+Gllg%GrrceZdZdZdZdZdZy)AsvImplementationzDAllows to select a library that we will use for testing performance.MODIN_ASV_USE_IMPL)modinpandasrNr+r,r-r$rr(r7r0rrrrsN"G!GGrrceZdZdZdZdZy)AsvDataSizeConfigz1Allows to override default size of data (shapes).MODIN_ASV_DATASIZE_CONFIGNr6r0rrrrs;)GGrrcbeZdZdZdZdZed dZed dZede ddffd Z xZ S) ProgressBarz(Whether or not to show the progress bar.MODIN_PROGRESS_BARFrNc&|jdy)zEnable ``ProgressBar`` feature.TNr]rs renablezProgressBar.enables  rc&|jdy)z Disable ``ProgressBar`` feature.FNrrs rdisablezProgressBar.disables rr[cf|rtjr tdt||y)z Set ``ProgressBar`` value only if synchronous benchmarking is disabled. Parameters ---------- value : bool Config value to set. z/ProgressBar isn't compatible with BenchmarkModeN) BenchmarkModerNrUrMr]r_s rr]zProgressBar.puts+ ]&&(NO O  ErrN) r+r,r-r$rr7r/rrboolr]r`ras@rrrsX2"GG    rrc>eZdZdZdZdZededdffd ZxZ S)rz5Whether or not to perform computations synchronously.MODIN_BENCHMARK_MODEFr[rNcf|rtjr tdt||y)z Set ``BenchmarkMode`` value only if progress bar feature is disabled. Parameters ---------- value : bool Config value to set. z/BenchmarkMode isn't compatible with ProgressBarN)rrNrUrMr]r_s rr]zBenchmarkMode.put s) [__&NO O  Er) r+r,r-r$rr7r/rr]r`ras@rrrs0?$GG    rrcReZdZdZdZdZdZed dZed dZ ed dZ y) LogModez.Set ``LogMode`` value if users want to opt-in.MODIN_LOG_MODE)rrenable_api_onlyrNc&|jdy)zEnable all logging levels.rNrrs rrzLogMode.enable!s rc&|jdy)zDisable logging feature.rNrrs rrzLogMode.disable&s  rc&|jdy)zEnable API level logging.rNrrs rrzLogMode.enable_api_only+s !"rr) r+r,r-r$rr(r7r/rrrr0rrrrsN8G6GG##rrcXeZdZdZdZdZededdffd Zedeffd Z xZ S) LogMemoryIntervalz@Interval (in seconds) to profile memory utilization for logging.MODIN_LOG_MEMORY_INTERVALr5r[rNcJ|dkrtd|t| |y)z Set ``LogMemoryInterval`` with extra checks. Parameters ---------- value : int Config value to set. rz0Log memory Interval should be > 0, passed value NrUrMr]r_s rr]zLogMemoryInterval.put7s, A:OPUwWX X  Erc<t|}|dkDsJd|S)zk Get ``LogMemoryInterval`` with extra checks. Returns ------- int rz!`LogMemoryInterval` should be > 0rMrN)rlog_memory_intervalrZs rrNzLogMemoryInterval.getEs+$gkm"Q&K(KK&""r r+r,r-r$rr7r/rr]rNr`ras@rrr1sKJ)GG     #C # #rrcXeZdZdZdZdZededdffd Zedeffd Z xZ S) LogFileSizez1Max size of logs (in MBs) to store per Modin job.MODIN_LOG_FILE_SIZE r[rNcJ|dkrtd|t| |y)z Set ``LogFileSize`` with extra checks. Parameters ---------- value : int Config value to set. rz-Log file size should be > 0 MB, passed value Nrr_s rr]zLogFileSize.putYs+ A:LUGTU U  Erc<t|}|dkDsJd|S)ze Get ``LogFileSize`` with extra checks. Returns ------- int rz`LogFileSize` should be > 0r)r log_file_sizerZs rrNzLogFileSize.getgs* q ?"?? rrras@rrrSsK;#GG     C  rrceZdZdZdZdZy)PersistentPicklez+Whether serialization should be persistent.MODIN_PERSISTENT_PICKLEFNr6r0rrrrus5'G GrrcLeZdZdZdZedeffd ZedefdZ xZ S)HdkLaunchParametersa Additional command line options for the HDK engine. Please visit OmniSci documentation for the description of available parameters: https://docs.omnisci.com/installation-and-configuration/config-parameters#configuration-parameters-for-omniscidb MODIN_HDK_LAUNCH_PARAMETERSrc t|}|jj}|j |j Dcic]\}}|j dd|c}}|Scc}}w)z Get the resulted command-line options. Decode and merge specified command-line options with the default one. Returns ------- dict Decoded and verified config value. -_)rMrNr}copyupdateitemsreplace)rcustom_parametersresultkeyr[rZs rrNzHdkLaunchParameters.getsl"GKM!!#((* ?)4GI&w  sAA** A76A7) r+r,r-r$rr/dictrNrr}r`ras@rrrsC,GD$SrrcXeZdZdZdZdZededdffd Zedeffd Z xZ S) MinPartitionSizez Minimum number of rows/columns in a single pandas partition split. Once a partition for a pandas dataframe has more than this many elements, Modin adds another partition. MODIN_MIN_PARTITION_SIZErr[rNcJ|dkrtd|t| |y)z Set ``MinPartitionSize`` with extra checks. Parameters ---------- value : int Config value to set. rz/Min partition size should be > 0, passed value Nrr_s rr]zMinPartitionSize.puts+ A:NugVW W  Erc<t|}|dkDsJd|S)zj Get ``MinPartitionSize`` with extra checks. Returns ------- int rz"`min_partition_size` should be > 0r)rmin_partition_sizerZs rrNzMinPartitionSize.gets+#W[]!A%K'KK%!!rrras@rrrsO)GG     "C " "rrceZdZdZdZdZy)TestReadFromSqlServerz,Set to true to test reading from SQL server.MODIN_TEST_READ_FROM_SQL_SERVERFNr6r0rrrrs6/GGrrceZdZdZdZdZy)TestReadFromPostgresz*Set to true to test reading from Postgres.MODIN_TEST_READ_FROM_POSTGRESFNr6r0rrrrs4-GGrrceZdZdZdZdZy)GithubCIz,Set to true when running Modin in GitHub CI.MODIN_GITHUB_CIFNr6r0rrrrs6GGrrc4eZdZdZdZdZedeefdZ y) ModinNumpyz7Set to true to use Modin's implementation of NumPy API. MODIN_NUMPYFrctSzGet a parameter sibling.)ExperimentalNumPyAPIrs rrEzModinNumpy._siblings $#rN r+r,r-r$rr7r/r&rBrEr0rrrrs.AGG$./$$rrc4eZdZdZdZdZedeefdZ y)r#z Set to true to use Modin's implementation of NumPy API. This parameter is deprecated. Use ``ModinNumpy`` instead. MODIN_EXPERIMENTAL_NUMPY_APIFrctSr")rrs rrEzExperimentalNumPyAPI._siblings rNr$r0rrr#r#s2 -GG./rr#c4eZdZdZdZdZedeefdZ y)RangePartitioningGroupbya Set to true to use Modin's range-partitioning group by implementation. Experimental groupby is implemented using a range-partitioning technique, note that it may not always work better than the original Modin's TreeReduce and FullAxis implementations. For more information visit the according section of Modin's documentation: TODO: add a link to the section once it's written. MODIN_RANGE_PARTITIONING_GROUPBYFrctSr")ExperimentalGroupbyImplrs rrEz!RangePartitioningGroupby._sibling+s '&rNr$r0rrr)r)s21GG'./''rr)c4eZdZdZdZdZedeefdZ y)r,z Set to true to use Modin's range-partitioning group by implementation. This parameter is deprecated. Use ``RangePartitioningGroupby`` instead. MODIN_EXPERIMENTAL_GROUPBYFrctSr")r)rs rrEz ExperimentalGroupbyImpl._sibling;s ('rNr$r0rrr,r,1s2 +GG(./((rr,ceZdZdZdZdZy)RangePartitioninga Set to true to use Modin's range-partitioning implementation where possible. Please refer to documentation for cases where enabling this options would be beneficial: https://modin.readthedocs.io/en/stable/flow/modin/experimental/range_partitioning_groupby.html MODIN_RANGE_PARTITIONINGFNr6r0rrr1r1Hs)GGrr1ceZdZdZdZdZy)CIAWSSecretAccessKeyzOSet to AWS_SECRET_ACCESS_KEY when running mock S3 tests for Modin in GitHub CI.AWS_SECRET_ACCESS_KEY foobar_secretNr6r0rrr4r4TsY%GGrr4ceZdZdZdZdZy)CIAWSAccessKeyIDzKSet to AWS_ACCESS_KEY_ID when running mock S3 tests for Modin in GitHub CI.AWS_ACCESS_KEY_ID foobar_keyNr6r0rrr8r8[sU!GGrr8ceZdZdZdZdZy) AsyncReadModea` It does not wait for the end of reading information from the source. It basically means, that the reading function only launches tasks for the dataframe to be read/created, but not ensures that the construction is finalized by the time the reading function returns a dataframe. This option was brought to improve performance of reading/construction of Modin DataFrames, however it may also: 1. Increase the peak memory consumption. Since the garbage collection of the temporary objects created during the reading is now also lazy and will only be performed when the reading/construction is actually finished. 2. Can break situations when the source is manually deleted after the reading function returns a result, for example, when reading inside of a context-block that deletes the file on ``__exit__()``. MODIN_ASYNC_READ_MODEFNr6r0rrr<r<bs&&GGrr<ceZdZdZdZdZdZy) ReadSqlEnginezEngine to run `read_sql`.MODIN_READ_SQL_ENGINEr)r ConnectorxNrr0rrr?r?zs#%GG&Grr?ceZdZdZdZdZdZy) LazyExecutiona Lazy execution mode. Supported values: `Auto` - the execution mode is chosen by the engine for each operation (default value). `On` - the lazy execution is performed wherever it's possible. `Off` - the lazy execution is disabled. MODIN_LAZY_EXECUTION)AutoOnOffrENrr0rrrCrCs%G#GGrrCceZdZdZdZdZy) DocModulez The module to use that will be used for docstrings. The value set here must be a valid, importable module. It should have a `DataFrame`, `Series`, and/or several APIs directly (e.g. `read_csv`). MODIN_DOC_MODULErNr6r0rrrIrIs!GGrrIceZdZdZdZdZy)DaskThreadsPerWorkerz"Number of threads per Dask worker.MODIN_DASK_THREADS_PER_WORKERrNr6r0rrrLrLs,-GGrrLctjDchc]:}t|tr(t |t r|j s |j<}}|jgdtjDchc]}|jds|}}||z }tjDcic]]}t|trKt |t r;|j s/|j#|j|j|j_}}||jz}|r[tjdt!|dkDrdnddd t!|dkDrd nd d zd j#t%|z|D]/}tj||j'dt(1ycc}wcc}wcc}w)z Check validity of environment variables. Look out for any environment variables that start with "MODIN_" prefix that are unknown - they might be a typo, so warn a user. )MODIN_PYTEST_CMDMODIN_PYTEST_DAILY_CMDMODIN_PYTEST_NO_COV_CMDMODIN_Nz"Found unknown environment variablers,z please check theiritsz spelling: rT)use_envvar_names)globalsvalues isinstancer& issubclassr is_abstractrrrr startswith_deprecation_descriptorkeysrOwarnlenr)sorteddeprecation_messagerR)obj valid_namesname found_namesunknown deprecatedfound_deprecateddepr_vars r _check_varsrms9##%  c4 s/ 0 KQ%'JJLD$//(2K4LKLK'G9##%4  c4 s/ 0 KK #  ' ' 3 S0004J4#Z__%66 0G q8Hb0QQR S#g,*:wFkR Siiw( ) %  x 4 4d 4 K   9M4s?GG G =A"Gr)Lr$rrrrOtextwraprtypingrr modin.configconfig modin_config packagingrpandas.util._decoratorsr&snowflake.snowpark.modin.config.pubsubrr r r r r rrrr3rr9r<r?rBrcrgrrrrrrrrrrrrrrrrrrrrrrrrrrrrr#r_r)r,r1r4r8r<r?rCrIrLrmr0rrrvsX$I   #'&)#&R+>S5Y 1 %8t /M +': -@s 1Q -* 3 W+ W+t! \ s\~('c(#(t# "&T" $)$ $*$+"+&"  s #"%C#"L()( &T)))&(t&+(+(%D>'d*#!#.#+##D%CD*<-D<~$"*$"N/d.T" $! $ +$ $0E*0, '/d'& (.T ($3H53/ +$ .S*'d0''c' 'c  #( .S' T r