ɯeiu zddlZddlZddlmZddlmZmZmZddlZ ddl m Z ddl m Z mZddlmZddlmZmZddlmZmZmZdd lmZmZdd lmZdd lmZmZm Z dd l!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'ejPd krddlm)Z)nddl*m)Z)ee+Z, ddede"de-de-fdZ.dZ/deee0e)e0fddfdZ1GddZ2y)N) getLogger)DictOptionalUnion) ColumnSum)build_expr_from_python_valwith_src_position)SnowparkClientExceptionMessages) add_api_calladjust_api_subcalls)$VALID_PYTHON_TYPES_FOR_LITERAL_VALUE LiteralTypepython_type_to_snow_type) publicapi quote_name)Column)ifflitwhen)DataType DecimalType DoubleType FloatType IntegerTypeLongType) )Iterablevaluedatatypeinclude_decimalreturnc pttttf}ttf}|r|tf}|tf}|duxs}t |t xrt |t xr t ||xsJt |txr t ||xs*t |ttt|dS)Nr) rrrrr isinstanceintboolfloattyper)rr r! int_types float_typess k/var/www/html/glpi_dashboard/venv/lib/python3.12/site-packages/snowflake/snowpark/dataframe_na_functions.py'_is_value_type_matching_for_na_functionr,/sh :>Ij)K , "K0    P uc " 0ud++ 08Y/  P ue $ JHk)J P h%=d5k%J1%M N O zDsubset should be a single column name, list or tuple of column namessubsetcv|7t|ts&t|ttfst t yyy)zD cic]"} tA| jB| jD$} } |D chc] } tA| } } g} | D]}|| vr$tGjH|| jK|jjM|d }t| |tNtPfr4tS|tTjVk(|jYd zd dd }ntS|jYd d dd }| j[|t]t_| Dcgc]}|j`c}d }|jjc||k\d }t;|d d |r|j4|_|Scc} wcc} wcc}w)a Returns a new DataFrame that excludes all rows containing fewer than a specified number of non-null and non-NaN values in the specified columns. Args: how: An ``str`` with value either 'any' or 'all'. If 'any', drop a row if it contains any nulls. If 'all', drop a row only if all its values are null. The default value is 'any'. If ``thresh`` is provided, ``how`` will be ignored. thresh: The minimum number of non-null and non-NaN values that should be in the specified columns in order for the row to be included. It overwrites ``how``. In each case: * If ``thresh`` is not provided or ``None``, the length of ``subset`` will be used when ``how`` is 'any' and 1 will be used when ``how`` is 'all'. * If ``thresh`` is greater than the number of the specified columns, the method returns an empty DataFrame. * If ``thresh`` is less than 1, the method returns the original DataFrame. subset: A list of the names of columns to check for null and NaN values. In each case: * If ``subset`` is not provided or ``None``, all columns will be included. * If ``subset`` is empty, the method returns the original DataFrame. Examples:: >>> df = session.create_dataframe([[1.0, 1], [float('nan'), 2], [None, 3], [4.0, None], [float('nan'), None]]).to_df("a", "b") >>> # drop a row if it contains any nulls, with checking all columns >>> df.na.drop().show() ------------- |"A" |"B" | ------------- |1.0 |1 | ------------- >>> # drop a row only if all its values are null, with checking all columns >>> df.na.drop(how='all').show() --------------- |"A" |"B" | --------------- |1.0 |1 | |nan |2 | |NULL |3 | |4.0 |NULL | --------------- >>> # drop a row if it contains at least one non-null and non-NaN values, with checking all columns >>> df.na.drop(thresh=1).show() --------------- |"A" |"B" | --------------- |1.0 |1 | |nan |2 | |NULL |3 | |4.0 |NULL | --------------- >>> # drop a row if it contains any nulls, with checking column "a" >>> df.na.drop(subset=["a"]).show() -------------- |"A" |"B" | -------------- |1.0 |1 | |4.0 |NULL | -------------- >>> df.na.drop(subset="a").show() -------------- |"A" |"B" | -------------- |1.0 |1 | |4.0 |NULL | -------------- See Also: :func:`DataFrame.dropna` N)anyallzhow ('z') should be 'any' or 'all'TFrBrzDataFrameNaFunctions.drop) _ast_stmtr@ len_subcallsr@)2 ValueErrorr5r:_session _ast_batchbindr exprdataframe_na_drop__pythonr>r?rr$r0r.variadicrargsaddr _set_ast_refdfcolumnslen_copy_without_astr uid_ast_idlimitr schemafieldsrnamer r DF_CANNOT_RESOLVE_COLUMN_NAMEkeyscolrrrmathnanis_nullappendrr _expressionwhere)r;r>r?r.r@stmtastr_new_dffielddf_col_type_dictcol_namenormalized_col_name_set is_na_columnsnormalized_col_nameis_nac col_counters r+dropzDataFrameNaFunctions.drop`s@ ?s.8vcU*EFG G' ??++66;;=D#DII$G$GNCCG!#)  &#&&* #*3::??+>+>+@&IFH-&+ #!KC.szz/B/B/DcJK OO ( ( 0 >__,,F  $XF >$'5LS[aF A:V)__668F !< =!%?? "c&k !__**1*NF (CRS T!%M"__33:: 5::&6   MS&Sz(';&S #&SM'> ,#&.>>9WW+-=-B-B-Doo))*=)O$%89Iz;R CKK%K,HH"' E  e | _ | St5| jCDcgc]}t|tDc}st9dtDd|jjFjHDcic]"}tK|jL|jN$}}i}| jD]=\} }tK| }||vr$tQjR||j7|||<?g}|jD]*\} }|jjU| } | |vr|| }tW|||rt|tXtZfrN|j]t_| t`jbk(| jez|| jg| |j]t_| jed || d jg| d thjkd | d |d |d tm||j]| |j]| -|jjo|| } tq| dd| Scc}wcc} wcc}wcc}w)a Returns a new DataFrame that replaces all null and NaN values in the specified columns with the values provided. Args: value: A scalar value or a ``dict`` that associates the names of columns with the values that should be used to replace null and NaN values in those columns. If ``value`` is a ``dict``, ``subset`` is ignored. If ``value`` is an empty ``dict``, the method returns the original DataFrame. subset: A list of the names of columns to check for null and NaN values. In each case: * If ``subset`` is not provided or ``None``, all columns will be included. * If ``subset`` is empty, the method returns the original DataFrame. include_decimal: Whether to allow ``Decimal`` values to fill in ``IntegerType`` and ``FloatType`` columns. Examples:: >>> df = session.create_dataframe([[1.0, 1], [float('nan'), 2], [None, 3], [4.0, None], [float('nan'), None]]).to_df("a", "b") >>> # fill null and NaN values in all columns >>> df.na.fill(3.14).show() --------------- |"A" |"B" | --------------- |1.0 |1 | |3.14 |2 | |3.14 |3 | |4.0 |NULL | |3.14 |NULL | --------------- >>> # fill null and NaN values in column "a" >>> df.na.fill(3.14, subset="a").show() --------------- |"A" |"B" | --------------- |1.0 |1 | |3.14 |2 | |3.14 |3 | |4.0 |NULL | |3.14 |NULL | --------------- >>> # fill null and NaN values in column "a" >>> df.na.fill({"a": 3.14}).show() --------------- |"A" |"B" | --------------- |1.0 |1 | |3.14 |2 | |3.14 |3 | |4.0 |NULL | |3.14 |NULL | --------------- >>> # fill null and NaN values in column "a" and "b" >>> df.na.fill({"a": 3.14, "b": 15}).show() -------------- |"A" |"B" | -------------- |1.0 |1 | |3.14 |2 | |3.14 |3 | |4.0 |15 | |3.14 |15 | -------------- >>> df2 = session.create_dataframe([[1.0, True], [2.0, False], [3.0, False], [None, None]]).to_df("a", "b") >>> df2.na.fill(True).show() ---------------- |"A" |"B" | ---------------- |1.0 |True | |2.0 |False | |3.0 |False | |NULL |True | ---------------- Note: If the type of a given value in ``value`` doesn't match the column data type (e.g. a ``float`` for :class:`~snowflake.snowpark.types.StringType` column), this replacement will be skipped in this column. Especially, * ``int`` can be filled in a column with :class:`~snowflake.snowpark.types.FloatType` or :class:`~snowflake.snowpark.types.DoubleType`, but ``float`` cannot filled in a column with :class:`~snowflake.snowpark.types.IntegerType` or :class:`~snowflake.snowpark.types.LongType`. See Also: :func:`DataFrame.fillna` NTFz.All keys in value should be column names (str)zDataFrameNaFunctions.fillz(All values in value should be in one of  typesrsrHzgInput value type doesn't match the target column data type, this replacement was skipped. Column Name: , Type: , Input Value: rErDrF)9r5r:rJrKrLr rMdataframe_na_fillrRrSr$dictitemsr0 value_maprQ_1r_2rr.rOrPrr!rTrCr^rIrVr rWrXvaluesr rZr[rr\r r r]r_r,rrrcrr`rarbas__loggerwarningr(selectr )r;rr.r@r!rfrgkventryr_ value_dictrkrhrirjnormalized_value_dictrn res_columnsr s r+fillzDataFrameNaFunctions.fillsrZ ' ??++66;;=D#DII$?$?FC OO ( ( 0%&!KKM@DAq!!S) # 1 1 3#$2588Q? @+399e<&#&&* #*3::??+>+>+@&IFH-&+ #!KC.szz/B/B/DcJK"1C  >__,,F  $XF eT "EJJLAq 1c*AB DJ:@Ah(E/AJA__668F !< =!%M$**, 1BC  :78@ //66  uzz "ENN 2  !#)//1 ?OHe",X"6 "*::5SS')9)>)>)@:? !"5 6  ? "2"8"8":! ( Hh//%%h/C00-h7:8_"(Y ,CD#**CKKM A5#NRR ($** # e < % #*/  "c(ec< OOFFNZP!!) /%e W  &&s+""3'C! (F'' t'DF$?aP YB B  s S; SS .'S% to_replacec| t|d}|r7|jjjj }t |j j|}|jj|jt|tr_|jD]K\}} |jj} t| j |t| j"| MnTt|t$r.|D](} |j&j} t| | *nt|j(|t|t$r.|D](} |j*j} t| | *nt|j,|t|t.r@d|j0_t|j0j4j|nVt|t$rFd|j0_|D]0} t|j0j4j| 2||_||jj8}nt|t.r|g}t;|dk(r;|jj=} t?| d|r|j@| _!| St|tr|} nt|tDtFfrt|tDtFfrZt;|t;|k7r#tIdt;|dt;|tK||D cic]\}} ||  } }} n|Dcic]}||} }n||i} | s;|jj=} t?| d|r|j@| _!| StM| jD cgc]'\}} t|tNxrt| tN)c} }stIdtNd |jjPjRDcic]"}tU|jV|jX$}}|Dchc] }tU|}}|D]*}||vst[j\||j_g}|jD]G\}}|jja|} ||vrd}| jD]\}}tc||| rgtc||| rY|| jen | tg|k(}| tgdn tg|}||ji||n ti||}{tjjmd |d |d |d to|d|d to| |3|jq| js|}|ju|$|ju| 7|ju| J|jjw||} ty| dd| Scc} }wcc}wcc} }wcc}wcc}w)a+ Returns a new DataFrame that replaces values in the specified columns. Args: to_replace: A scalar value, or a list of values or a ``dict`` that associates the original values with the replacement values. If ``to_replace`` is a ``dict``, ``value`` and ``subset`` are ignored. To replace a null value, use ``None`` in ``to_replace``. To replace a NaN value, use ``float("nan")`` in ``to_replace``. If ``to_replace`` is empty, the method returns the original DataFrame. value: A scalar value, or a list of values for the replacement. If ``value`` is a list, ``value`` should be of the same length as ``to_replace``. If ``value`` is a scalar and ``to_replace`` is a list, then ``value`` is used as a replacement for each item in ``to_replace``. subset: A list of the names of columns in which the values should be replaced. If ``cols`` is not provided or ``None``, the replacement will be applied to all columns. If ``cols`` is empty, the method returns the original DataFrame. include_decimal: Whether to allow ``Decimal`` values to replace ``IntegerType`` and ``FloatType`` values. Examples:: >>> df = session.create_dataframe([[1, 1.0, "1.0"], [2, 2.0, "2.0"]], schema=["a", "b", "c"]) >>> # replace 1 with 3 in all columns >>> df.na.replace(1, 3).show() ------------------- |"A" |"B" |"C" | ------------------- |3 |3.0 |1.0 | |2 |2.0 |2.0 | ------------------- >>> # replace 1 with 3 and 2 with 4 in all columns >>> df.na.replace([1, 2], [3, 4]).show() ------------------- |"A" |"B" |"C" | ------------------- |3 |3.0 |1.0 | |4 |4.0 |2.0 | ------------------- >>> # replace 1 with 3 and 2 with 3 in all columns >>> df.na.replace([1, 2], 3).show() ------------------- |"A" |"B" |"C" | ------------------- |3 |3.0 |1.0 | |3 |3.0 |2.0 | ------------------- >>> # the following line intends to replaces 1 with 3 and 2 with 4 in all columns >>> # and will give [Row(3, 3.0, "1.0"), Row(4, 4.0, "2.0")] >>> df.na.replace({1: 3, 2: 4}).show() ------------------- |"A" |"B" |"C" | ------------------- |3 |3.0 |1.0 | |4 |4.0 |2.0 | ------------------- >>> # the following line intends to replace 1 with "3" in column "a", >>> # but will be ignored since "3" (str) doesn't match the original data type >>> df.na.replace({1: "3"}, ["a"]).show() ------------------- |"A" |"B" |"C" | ------------------- |1 |1.0 |1.0 | |2 |2.0 |2.0 | ------------------- Note: If the type of a given value in ``to_replace`` or ``value`` doesn't match the column data type (e.g. a ``float`` for :class:`~snowflake.snowpark.types.StringType` column), this replacement will be skipped in this column. Especially, * ``int`` can replace or be replaced in a column with :class:`~snowflake.snowpark.types.FloatType` or :class:`~snowflake.snowpark.types.DoubleType`, but ``float`` cannot replace or be replaced in a column with :class:`~snowflake.snowpark.types.IntegerType` or :class:`~snowflake.snowpark.types.LongType`. * ``None`` can replace or be replaced in a column with any data type. See Also: :func:`DataFrame.replace` NTFrzDataFrameNaFunctions.replacez+>+@&IFH-&+ #!KC.szz/B/B/DcJK"1C  >__,,F  $XF v;! __668F !? @!%M j$ '$K  T5M 2%$/z?c%j0$":/uSZLB 58 E4J"KDAq1a4"KK"K1;)>)@   "2"8"8":# ( Hh//%%h/C22 "-"3"3"5JC> (7B (7 14 s{{}#S/5:]D E  )4&NN4?!%dM!:"  JJRT%%-JmC5c T,,17(4;-I%0( ) 3 3C 8 < rs  ((FWT E,77v( H  "  :K 5HU3 3E-F$G5D5G G r-