ɯei-QddlmZddlZddlZddlZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlZej&dkrddlmZnddlmZd Zd Zd d gZd dgZej4eZGddeZy)) annotationsN) RawIOBaseUnsupportedOperationBufferedReaderSEEK_SETSEEK_ENDSEEK_CUR)SNOWFLAKE_PATH_PREFIXES)Sequence)get_active_session) )IterablezISnowflakeFile currently doesn't support write APIs in local testing mode.z/Not yet supported in UDF and Stored Procedures.rrbwwbcZeZdZdZ d!ddd d"fdZe d!dd d#dZed$d%dZd&d Zd&d Z d&d Z d'd Z d&d Z d&dZ d(dZd&dZd)dZd*d+dZd*d+dZd)dZd,dZd'dZd'dZd*d+dZd*d-dZefd.dZd)dZd(dZd/d0dZd'dZd)dZd1d Z xZ!S)2 SnowflakeFilea SnowflakeFile provides an interface to operate on files as Python IOBase-like objects in UDFs and stored procedures. SnowflakeFile supports most operations supported by Python IOBase objects. A SnowflakeFile object can be used as a Python IOBase object. The constructor of this class is not supposed to be called directly. Call :meth:`~snowflake.snowpark.file.SnowflakeFile.open` to create a read-only SnowflakeFile object, and call :meth:`~snowflake.snowpark.file.SnowflakeFile.open_new_result` to create a write-only SnowflakeFile object. This class is used to read and write files in UDFs and stored procedures. On Snowflake, it is used to read and write stage files. It also supports Python IOBase and BufferedBase methods such as :meth:`read`, :meth:`write`, :meth:`close`. To read from a staged file, use the following API: Example:: >>> from snowflake.snowpark.files import SnowflakeFile >>> from snowflake.snowpark.functions import udf >>> @udf ... def read_file(url: str) -> str: ... file = SnowflakeFile.open(url, "r") ... return file.read() To write to a staged file first write to a result file via the following example. The result file will return as a scoped URL which can be copied to a permanent stage with `copy files `_ or read directly via another call to SnowflakeFile.read() in another UDF invocation. See `writing files from Snowpark Python UDFs `_ for more details. Example:: >>> from snowflake.snowpark.files import SnowflakeFile >>> from snowflake.snowpark.functions import udf >>> @udf ... def write_file(content: str) -> str: ... file = SnowflakeFile.open_new_result("w") ... file.write(content) ... return file # file must be returned to be accessible These examples are using the client, but this same pattern can be used inside SQL-defined UDFs. We provide a local implementation of SnowflakeFile to aid in local testing. This currently supports using read APIs on relative paths, mocked stages (sessions in local testing mode that aren't connected to a real stage), and Snowflake stages. Scoped and Stage URLs (https://) are not yet supported. Note: 1. All of the implementation in this file is for local testing purposes. 2. There may be slight implementation differences between local testing and Snowflake execution environments, which we call out below when we can. If any issues or these differences block your testing workflow, please file a bug report at https://github.com/snowflakedb/snowpark-python/issues. 3. UDF implementation is dependent on the Snowflake release. Therefore, this documentation may not always be up to date. FTrequire_scoped_urlfrom_result_apic t |||_||_||_||_d|_d|_d|_d}d|_ |jjdr td|jjttrdnd|_|j |_d|_|jr|t"vr||dk(rdnd}t%t'|j|j| ||_t'|jd }|j+dt,|_|j/y|jr|t"vrxt1j2j5|j|_|j(j+dt,|_|j(j+dt6y|t8vr&t'|j|j|_yy) Nirzhttps://z,Scoped and Stage URLs are not yet supported.TFrzutf-8)encodingr)super__init___file_location_mode_is_owner_file_require_scoped_urlbufferrerrors_pos startswith ValueErrortupler _is_stage_file_is_local_file _file_size _READ_MODESropen _file_streamseekrcloser file get_streamr _WRITE_MODES) self file_locationmode is_owner_filerr_DEFAULT_READ_BUFFER_SIZEr temp_file __class__s Z/var/www/html/glpi_dashboard/venv/lib/python3.12/site-packages/snowflake/snowpark/files.pyrzSnowflakeFile.__init___s #0 +#5    %.!    ) )* 5KL L""--e4K.LM   #'"5"55   4;#6"&#+w4H .T(($**xH)!D T00$7I'nnQ9DO OO   T[%8 2 4 9 9 D D##!D #//44QADO    " "1h / \ ! $T%8%8$** ED "rcH|tvrtd|d|||||S)af Used to create a :class:`~snowflake.snowpark.file.SnowflakeFile` which can only be used for read-based IO operations on the file. In UDFs and Stored Procedures, the object works like a read-only Python IOBase object and as a wrapper for an IO stream of remote files. All files are accessed in the context of the UDF owner (with the exception of caller's rights stored procedures which use the caller's context). UDF callers should use scoped URLs to allow the UDF to access their files. By accepting only scoped URLs the UDF owner can ensure the UDF caller had access to the provided file. Removing the requirement that the URL is a scoped URL (require_scoped_url=False) allows the caller to provide URLs that may be only accessible by the UDF owner. is_owner_file is marked for deprecation. For Snowflake release 7.8 and onwards please use require_scoped_url instead. Args: file_location: scoped URL, file URL, or string path for files located in a stage mode: A string used to mark the type of an IO stream. Supported modes are "r" for text read and "rb" for binary read. is_owner_file: (Deprecated) A boolean value, if True, the API is intended to access owner's files and all URI/URL are allowed. If False, the API is intended to access files passed into the function by the caller and only scoped URL is allowed. require_scoped_url: A boolean value, if True, file_location must be a scoped URL. A scoped URL ensures that the caller cannot access the UDF owners files that the caller does not have access to. Invalid mode 'z;' for SnowflakeFile.open. Supported modes are 'r' and 'rb'.r;)r*r%)clsr3r4r5rs r9r+zSnowflakeFile.opens?6 { " &ab  4CU  r:c|tvrtd|d|tjj|ddS)a> Used to create a :class:`~snowflake.snowpark.file.SnowflakeFile` which can only be used for write-based IO operations. UDFs/Stored Procedures should return the file to materialize it, and it is then made accessible via a scoped URL returned in the query results. In UDFs and Stored Procedures, the object works like a write-only Python IOBase object and as a wrapper for an IO stream of remote files. Args: mode: A string used to mark the type of an IO stream. Supported modes are "w" for text write and "wb" for binary write. r=zF' for SnowflakeFile.open_new_result. Supported modes are 'w' and 'wb'.rTr)r1r%tempfileNamedTemporaryFilename)r>r4s r9open_new_resultzSnowflakeFile.open_new_resultsP | # &lm   ' ' ) . .     r:cX|jtvrtd|jy)zo Internal function to validate read mode of the file object before performing an IO operation. zNot readable mode=N)rr*rr2s r9_raise_if_not_readz SnowflakeFile._raise_if_not_reads, ::[ (&);DJJ<'HI I )r:cX|jtvrtd|jy)zo Internal function to validate write mode of the file object before performing a IO operation. zNot writable mode=N)rr1rrEs r9_raise_if_not_writez!SnowflakeFile._raise_if_not_writes, ::\ )&);DJJ<'HI I *r:cF|jjr tdy)zq Internal function to validate open status of the file object before performing an IO operation. zI/O operation on closed file.N)r,closedr%rEs r9_raise_if_closedzSnowflakeFile._raise_if_closeds$    # #<= = $r:ct|}|dk(ry|j|}t|}|xj|jz c_||d||jS)z Internal function to read bytes into a pre-allocated, writable bytes-like object buffer. This is used by readinto and readinto1 methods. rN)lenread1 memoryviewr#nbytes)r2b buffer_lencontentsizes r9_read_into_bufferzSnowflakeFile._read_into_buffersV V ?**Z('" T[[  +:{{r:c8|jjy)z See https://docs.python.org/3/library/io.html#io.IOBase.close Closes the underlying IO Stream of the SnowflakeFile. N)r,r.rEs r9r.zSnowflakeFile.closes !r:c8|jtd)z Not yet supported in UDF and Stored Procedures. See https://docs.python.org/3/library/io.html#io.BufferedIOBase.detach z)Detaching stream from file is unsupported)rKrrEs r9detachzSnowflakeFile.detachs "#NOOr:c8|jtd)zd Getting a file descriptor number is not supported in Snowflake. Raises an OSError. z*This object does not use a file descriptor)rKOSErrorrEs r9filenozSnowflakeFile.filenos BCCr:c$|jy)z= Fail if the stream is closed. Does nothing. NrKrEs r9flushzSnowflakeFile.flushs  r:c$|jy)zo Returns False, file streams in stored procedures and UDFs are never interactive in Snowflake. Fr]rEs r9isattyzSnowflakeFile.isattys r:cr|j|j|jr&|jjj |}nF|j r:|jj |}|jdk(r|j}|xjtz c_ |S)am From https://docs.python.org/3/library/io.html#io.RawIOBase.read Read up to size bytes from the object and return them. As a convenience, if size is unspecified or -1, all bytes until EOF are returned. Fewer than size bytes may be returned. If 0 bytes are returned, and size was not 0, this indicates end of file. r) rKrFr(r,rawreadr'rdecoder#rMr2rTrSs r9rczSnowflakeFile.reads  !   ''++006G  '',,T2GzzS !..* S\! r:c|j|j|jrZ|jdk(r |j |j }nR|jdk(rC|j j|}n'|jr|j j|}|xjtz c_ |S)uQ From https://docs.python.org/3/library/io.html#io.BufferedIOBase.read1 Read and return up to size bytes, with at most one call to the underlying raw stream’s read() (or readinto()) method. If size is -1 (the default), an arbitrary number of bytes are returned (more than zero unless EOF is reached). rr) rKrFr(rrcencoder,rNr'r#rMres r9rNzSnowflakeFile.read12s  !   zzS ))D/002t#++11$7  ''--d3G S\! r:cV|j|jjS)z From https://docs.python.org/3/library/io.html#io.IOBase.readable Returns whether or not the stream is readable. )rKr,readablerEs r9rizSnowflakeFile.readableF%   ))++r:c$|jdS)z From https://docs.python.org/3/library/io.html#io.RawIOBase.readall Read and return all the bytes from the stream until EOF, using multiple calls to the stream if necessary. )rcrEs r9readallzSnowflakeFile.readallOs yy   r:cb|j|j|jrF|jdk(r|j |S|j j j|}n'|jr|j j|}|xjz c_ |S)z From https://docs.python.org/3/library/io.html#io.RawIOBase.readinto Read bytes into a pre-allocated, writable bytes-like object b, and return the number of bytes read. For example, b might be a bytearray. r) rKrFr(rrUr,rbreadintor'r#r2rQrTs r9rozSnowflakeFile.readintoYs  !   zzS --a00$$((11!4D  $$--a0D T  r:cN|j|j|jr<|jdk(r|j |S|j j |}n'|jr|j j |}|xjz c_|S)z From https://docs.python.org/3/library/io.html#io.BufferedIOBase.readinto1 Read bytes into a pre-allocated, writable bytes-like object b. Return the number of bytes read. r) rKrFr(rrUr, readinto1r'r#rps r9rrzSnowflakeFile.readinto1ks  !   zzS --a00$$..q1D  $$..q1D T  r:cr|j|j|jr&|jjj |}nF|j r:|jj |}|jdk(r|j}|xjtz c_ |S)z From https://docs.python.org/3/library/io.html#io.IOBase.readline Read and return one line from the stream. If size is specified, at most size bytes will be read. r) rKrFr(r,rbreadliner'rrdr#rMres r9rtzSnowflakeFile.readline|s  !   ''++44T:G  ''006GzzS !..* S\! r:c|j|j|jr&|jjj |}nS|j rG|jj |}|jdk(r|Dcgc]}|j}}|xjtdDz c_ |Scc}w)u From https://docs.python.org/3/library/io.html#io.IOBase.readlines Read and return a list of lines from the stream. hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint. hint values of 0 or less, as well as None, are treated as no hint. Note that it’s already possible to iterate on file objects using for line in file: ... without calling file.readlines(). rc32K|]}t|ywN)rM).0lines r9 z*SnowflakeFile.readlines..s7tT7s) rKrFr(r,rb readlinesr'rrdr#sum)r2hintrSrys r9r{zSnowflakeFile.readliness  !   ''++55d;G  ''11$7GzzS 5<=T4;;=== S7w777 >s C c~|j|j|tk(r|}nN|tk(r|jj |z}n'|t k(r|j|z}ntd||dkrtd|||_ |jj|jtS)a See https://docs.python.org/3/library/io.html#io.IOBase.seek Move the stream position to a new location given an offset and a starting position. SEEK_SET/0 indicates a position relative to the start of the file. SEEK_CUR/1 indicates a position relative to the current stream position. SEEK_END/2 indicates a position relative to the end of the file. Only supported in read mode. Returns the new stream position. Not supported in write mode. zUnsupported whence value rzNegative seek position ) rKrFrr r,tellrr)NotImplementedErrorr%r#r-)r2offsetwhenceposs r9r-zSnowflakeFile.seeks  ! X C x ##((*V3C x //F*C%(A&&JK K 76se<= =   %%dii::r:cF|j|jtvS)z See https://docs.python.org/3/library/io.html#io.IOBase.seekable Returns whether or not the stream is seekable. )rKrr*rEs r9seekablezSnowflakeFile.seekables zz[((r:cZ|j|j|jS)z See https://docs.python.org/3/library/io.html#io.IOBase.tell Gets the current stream position. Not supported in write mode. )rKrFr#rEs r9rzSnowflakeFile.tells&  !yyr:c`|j|jtt)zA Not yet supported in UDF and Stored Procedures. )rKrHr_DEFER_IMPLEMENTATION_ERR_MSG)r2rTs r9truncatezSnowflakeFile.truncates(    "!"?@@r:c tt)a See https://docs.python.org/3/library/io.html#io.RawIOBase.write Write the given bytes-like object, b, to the underlying raw stream, and return the number of bytes-like objects written (e.g. unicode characters provided to a text input count as 1). The number of bytes should equal the input bytes, since all bytes are written directly to the stream. Local testing support is not implemented yet. r_WRITE_MODE_ERR_MSG)r2rQs r9writezSnowflakeFile.writes""566r:cV|j|jjS)z See https://docs.python.org/3/library/io.html#io.IOBase.writable Returns whether or not the stream is writable. )rKr,writablerEs r9rzSnowflakeFile.writablerjr:c tt)a+ From https://docs.python.org/3/library/io.html#io.IOBase.writelines Write a list of lines to the stream. Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end. Local testing support is not implemented yet. r)r2liness r9 writelineszSnowflakeFile.writeliness""566r:)rF) r3strr4rr5boolrrrrreturnNone) r3rr4rr5rrrrr)r)r4rrr)rr)rQzbytes | bytearray | array.arrayrint)rr)rr)rl)rTrrr )rr )r}rrzlist[Sequence])rrrrrrrw)rTz int | Nonerr)rzIterable[str] | list[str]rr)"__name__ __module__ __qualname____doc__r classmethodr+rCrFrHrKrUr.rXr[r^r`rcrNrirmrorrrtr{rr-rrrrrr __classcell__)r8s@r9rr,sS0j# ;F $( %;F;F;F ;F ! ;F;F ;Fz#  $(        !       D  (JJ> "PD *(, $"",/7;4)A7,7r:r) __future__rarraysysr@iorrrrrr "snowflake.snowpark._internal.utilsr typingr snowflake.snowpark.contextr logging version_inforcollections.abcrrr*r1 getLoggerr_loggerrr:r9rs # 9 v(P!RDk T{ '  H %F7IF7r: