*jLM UddlmZddlZddlZddlmZmZddlZddlZ ddl m Z ddl m Z ddlmZddlmZddlmZddlZdd lmZmZmZmZmZdd lmZmZej@e jBe jDe jFe jHd ejJe jLe jNe jPe jRd ejTe jVe jXd ejZe.e.d iZ/de0d<eddd dZ1dd!dZ2d"dZ3d#dZ4d$dZ5d#dZ6d%dZ7d&dZ8dd d'dZ9e d( d)dZ:e d( d*dZ:e d( d+dZ: d d+dZ:y),) annotationsN)Anyoverload)using_string_dtype)import_optional_dependency)Pandas4Warning) set_module)find_stack_level)BufferColumnColumnNullType DataFrame DtypeKind) ArrowCTypes Endianness) @)rr)rzdict[DtypeKind, dict[int, Any]] _NP_DTYPESzpandas.api.interchangect|tjr|St|dr1 t dd} |j |j | St|d s td tjd ttt!|j#| | S#|j$r}t||d}~wwxYw#t$r'tjdttYwxYw) a Build a ``pd.DataFrame`` from any DataFrame supporting the interchange protocol. .. note:: For new development, we highly recommend using the Arrow C Data Interface alongside the Arrow PyCapsule Interface instead of the interchange protocol. From pandas 3.0 onwards, `from_dataframe` uses the PyCapsule Interface, only falling back to the interchange protocol if that fails. From pandas 4.0 onwards, that fallback will no longer be available and only the PyCapsule Interface will be used. .. warning:: Due to severe implementation issues, we recommend only considering using the interchange protocol in the following cases: - converting to pandas: for pandas >= 2.0.3 - converting from pandas: for pandas >= 3.0.0 Parameters ---------- df : DataFrameXchg Object supporting the interchange protocol, i.e. `__dataframe__` method. allow_copy : bool, default: True Whether to allow copying the memory to perform the conversion (if false then zero-copy approach is requested). Returns ------- pd.DataFrame A pandas DataFrame built from the provided interchange protocol object. See Also -------- pd.DataFrame : DataFrame class which can be created from various input data formats, including objects that support the interchange protocol. Examples -------- >>> df_not_necessarily_pandas = pd.DataFrame({"A": [1, 2], "B": [3, 4]}) >>> interchange_object = df_not_necessarily_pandas.__dataframe__() >>> interchange_object.column_names() Index(['A', 'B'], dtype='str') >>> df_pandas = pd.api.interchange.from_dataframe( ... interchange_object.select_columns_by_name(["A"]) ... ) >>> df_pandas A 0 1 1 2 These methods (``column_names``, ``select_columns_by_name``) should work for any dataframe library which implements the interchange protocol. __arrow_c_stream__pyarrowz14.0.0) min_version)zero_copy_onlyNzConversion using Arrow PyCapsule Interface failed due to missing PyArrow>=14 dependency, falling back to (deprecated) interchange protocol. We recommend that you install PyArrow>=14.0.0.) stacklevel __dataframe__z#`df` does not support __dataframe__aThe Dataframe Interchange Protocol is deprecated. For dataframe-agnostic code, you may want to look into: - Arrow PyCapsule Interface: https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html - Narwhals: https://github.com/narwhals-dev/narwhals ) allow_copy) isinstancepdrhasattrrtable to_pandas ArrowInvalid RuntimeError ImportErrorwarningswarn UserWarningr ValueErrorr_from_dataframer)dfrpaes HC:\FundamentalWorkbench\vendor\pandas\core\interchange\from_dataframe.pyfrom_dataframer1)sv"bll# r'( -+I8LB -xx|--^-LL 2 '>?? MM A #%   J/J ?? -"1o1, -  MM#+-   s( C!B22C C  C-DDcbg}|jD]}t|}|j| |st|dkDr t d|s t|}n-t|dk(r|d}nt j |ddd}|jjdd}|||_ |S) a Build a ``pd.DataFrame`` from the DataFrame interchange object. Parameters ---------- df : DataFrameXchg Object supporting the interchange protocol, i.e. `__dataframe__` method. allow_copy : bool, default: True Whether to allow copying the memory to perform the conversion (if false then zero-copy approach is requested). Returns ------- pd.DataFrame rzHTo join chunks a copy is required which is forbidden by allow_copy=FalserTF)axis ignore_indexcopyz pandas.indexN) get_chunksprotocol_df_chunk_to_pandasappendlenr&r!concatmetadatagetindex)r-r pandas_dfschunk pandas_df index_objs r0r,r,s J/6 )$! #j/A- V   /3 ZA qM IIjqt%P  5I#  ci}g}|jD]6}t|tstd|d||vrtd|d|j |}|j d}|t jt jt jt jfvrt|\||<}n~|t jk(rt|\||<}nY|t jk(rt|\||<}n4|t j k(rt#|\||<}nt%d|d|j'|9t)j*|}||j,d<|S)z Convert interchange protocol chunk to ``pd.DataFrame``. Parameters ---------- df : DataFrameXchg Returns ------- pd.DataFrame zColumn z is not a stringz is not uniquerz Data type z not handled yet_INTERCHANGE_PROTOCOL_BUFFERS) column_namesr strr+get_column_by_namedtyperINTUINTFLOATBOOLprimitive_column_to_ndarray CATEGORICALcategorical_column_to_seriesSTRINGstring_column_to_ndarrayDATETIMEdatetime_column_to_ndarrayNotImplementedErrorr8r!rattrs)r-columnsbuffersnamecolrHbufr@s r0r7r7s]!GG!$$wtf,<=> > 7?wtfN;< <##D) !   MM NN OO NN   "=S!A GDM3 i++ +!=c!B GDM3 i&& &!9#!> GDM3 i(( (!;C!@ GDM3% 5'9I&JK Ks/"2 W%I7>IOO34 rBc|j}|d\}}t|||j|j}t |||d}||fS)aJ Convert a column holding one of the primitive dtypes to a NumPy array. A primitive type is one of: int, uint, float, bool. Parameters ---------- col : Column Returns ------- tuple Tuple of np.ndarray holding the data and the memory owner object that keeps the memory alive. dataoffsetlengthvalidity) get_buffersbuffer_to_ndarrayr^size set_nulls)rYrW data_buff data_dtyper\s r0rMrMsZ ooG#FOIz :cjj D T3 3 4D =rBc|j}|ds td|d}t|dr tj|j }n td|j }|d\}}t|||j|j}t|dkDr||t|z}n|}tj|||d  } tj| } t| ||d } | |fS) a Convert a column holding categorical data to a pandas Series. Parameters ---------- col : Column Returns ------- tuple Tuple of pd.Series holding the data and the memory owner object that keeps the memory alive. is_dictionaryz-Non-dictionary categoricals not supported yet categories_colz}Interchanging categorical columns isn't supported yet, and our fallback of using the `col._col` attribute (a ndarray) failed.r\r]r is_ordered)riorderedr`)describe_categoricalrTr"nparrayrjrarbr^rcr9r! CategoricalSeriesrd) rY categorical cat_columnrirW codes_buff codes_dtypecodesvaluescatr\s r0rOrOs**K  '!"QRR\*Jz6"XXjoo. ! M  ooG%foJ K 388: E  :EC O34 ..:{D T3 3 4D =rBcv|j\}}|tjtjtjfvrt |d|j }|dsJd|d\}}|jdtjtjfvsJtjdtjtjf}t!||d|j"}|d\}} t!|| |j$|j'd z} d } |tjtjfvr;|d } | 4| \} }t!| ||j$|j'} |dk(r| } d g|j'z}t)|j'D]M}| | |rt*j,||<|| || |d z}t/|}|j1d }|||<Ot3rt5j6|d}||fSt+j8|d}||fS)a Convert a column holding string data to a NumPy array. Parameters ---------- col : Column Returns ------- tuple Tuple of np.ndarray holding the data and the memory owner object that keeps the memory alive. z3 null kind is not yet supported for string columns.offsetsz#String buffers must contain offsetsr\rrr]rNr`zutf-8)encodingrFrHobject) describe_nullr NON_NULLABLE USE_BITMASK USE_BYTEMASKrTrarHrrP LARGE_STRINGrrJUINT8rNATIVErbbufsizer^rcrangernnanbytesdecoderr!rqasarray)rY null_kind sentinel_valrWre_rfr\ offset_buff offset_dtyperznull_posr` valid_buff valid_dtypestr_listiunits str_bytesstringress r0rQrQ-sY"//I|##""## "kL M  ooG 9 DDD 6?LIq 99Q<       J Y 1YEVEV WD!( 2K \#**SXXZ!^GH^//1L1LMM:&  &. #J (K 388:Hq $9+/#((*)>$+// 4@L!OE7:P"QRR ,,--l;KA~!V#VV! ' 2oo** HHJ  2$$VZZ8 9 +  zz#{{ JJ&9,1 2FNN;4O  A:<<((fY(G Gxx+..rBcyNr\rYr`allow_modify_inplaces r0rdrds rBcyrrrs r0rdrds rBcyrrrs r0rdrd"s !rBcP||S|j\}}d}|tjk(rtj||k(}n|tj tj fvr5|\}}t|||j|j}|dk(r6|}n2|tjtjfvrntd|d|/tj|r|s|j} d||<|S|S#t $r|j#t$}d||<Y|SwxYw)a Set null values for the data according to the column null kind. Parameters ---------- data : np.ndarray or pd.Series Data to set nulls in. col : Column Column object that describes the `data`. validity : tuple(Buffer, dtype) or None The return value of ``col.buffers()``. We do not access the ``col.buffers()`` here to not take the ownership of the memory of buffer objects. allow_modify_inplace : bool, default: True Whether to modify the `data` inplace when zero-copy is possible (True) or always modify a copy of the `data` (False). Returns ------- np.ndarray or pd.Series Data with the nulls being set. Nr]rz Null kind r)rr USE_SENTINELr!rqrrrbr^rcrUSE_NANrTrnanyr5 TypeErrorrfloat) r\rYr`rrrrrrs r0rdrd+s-6 !//I|HN///99T?l2 ~11>3N3NO O"* K$  CJJsxxz  1  yH ~22N4J4JK K !Jyk9O"PQQx 0#99;D "!DN K4K ";;u%D!DN K "s5C>>#D%$D%)T)rboolreturn pd.DataFrame)r- DataFrameXchgrrrr)r-rrr)rYr rztuple[np.ndarray, Any])rYr rztuple[pd.Series, Any])rzpd.Series | np.ndarray)rYr rz"tuple[np.ndarray | pd.Series, Any]) rr rHztuple[DtypeKind, int, str, str]r_intr^rr np.ndarray).) r\rrYr r`5tuple[Buffer, tuple[DtypeKind, int, str, str]] | Nonerrrr) r\ pd.SeriesrYr r`rrrrr) r\np.ndarray | pd.SeriesrYr r`rrrrr); __future__rrrtypingrrr(numpyrnpandas._configrpandas.compat._optionalr pandas.errorsrpandas.util._decoratorsr pandas.util._exceptionsr pandasr!*pandas.core.interchange.dataframe_protocolr r r rrrpandas.core.interchange.utilsrrrIint8int16int32int64rJuint8uint16uint32rrKfloat32float64rLrr__annotations__r1r,r7rMrOrQrrSrbrdrrBr0rsG" ->(.4 MMrwwBHH"((I NNbiiRYYBIIN OO"**"**5 NN& / + $%^&^B$N)X61h\~OB"T ;/ ;/ *;/  ;/  ;/  ;/| "%   D     "%   D     "% ! ! !D! !  ! !"& : : :D: :  :rB