*jЃ<dZddlmZddlmZddlmZmZddlZ ddlm Z ddl m Z ddl mZdd lmZdd lmZmZmZmZdd lmZmZmZmZmZmZdd lmZmZdd l m!Z!m"Z"ddl#m$Z$ddl%m&Z&m'Z(ddl)m*Z*ddl+m,Z,ddl-m.Z/m0Z0m1Z1m2Z2ddl3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAddlBmCZCmDZDerddlEmFZFmGZGddlHmIZImJZJmKZKmLZLdddd d1dZM d2dZN d3dZO d4dZPddd d5dZQ d6d ZRd7d!ZSd8d9d"ZTd:d#ZU d;d$ZVdd(ZZd)Z[ d? d@d*Z\dAd+Z] dBd,Z^ dCd-Z_ dDd.Z` dEd/Za dF dGd0Zby)Hz~ Functions for preparing various inputs passed to the DataFrame or Series constructors before passing them to a BlockManager. ) annotations)abc) TYPE_CHECKINGAnyN)ma)using_string_dtype)lib)astype_is_view)"construct_1d_arraylike_from_scalar dict_compatmaybe_cast_to_datetimemaybe_convert_platform)is_1d_only_ea_dtypeis_integer_dtype is_list_likeis_named_tupleis_object_dtype is_scalar)BaseMaskedDtypeExtensionDtype) ABCDataFrame ABCSeries)isna) algorithmscommon)ExtensionArray) StringDtype)array extract_arrayrange_to_ndarraysanitize_array) DatetimeIndexIndex MultiIndexTimedeltaIndex default_index ensure_indexget_objs_combined_axismaybe_sequence_to_range union_indexes)BlockPlacementensure_block_shape new_block new_block_2d) create_block_manager_from_blocks'create_block_manager_from_column_arrays)HashableSequence) ArrayLikeDtypeObjManagernptT)dtypeverify_integrity consolidatec|r*| t|}n t|}t|||\}}nt|}|Dcgc]}t|d}}dgt |z}|D]S}t |t jtfr'|jdk7st |t |k7sJtdt|}t |t |k7r td||g} t|| ||Scc}w)zs Segregate Series based on type and coerce into matrices. Needs to handle a lot of exceptional cases. NT extract_numpyzYArrays must be 1-dimensional np.ndarray or ExtensionArray with length matching len(index)z#len(arrays) must match len(columns))r9refs) _extract_indexr' _homogenizerlen isinstancenpndarrayrndim ValueErrorr0) arrayscolumnsindexr7r8r9r>xarraxess DC:\FundamentalWorkbench\vendor\pandas\core\internals\construction.py arrays_to_mgrrN`s ="6*E 'E#65%8 U#@FG1-6GvF #CsRZZ$@A88q=s8s5z) 6 7#G 7|s6{">?? U D 2+D /HsC9c.tj|}|tt|}n t |}| t |}t ||\}}t |||t|\}}||}t||||}|r|jd}|S)zA Extract from a masked rec array and create the manager. r7Tdeep) rgetdatar&rAr' to_arraysreorder_arraysrNcopy) datarIrHr7rVfdatarG arr_columnsmgrs rMrec_array_to_mgrr[s JJt E }c%j)U#w'#E73FK)gs5zRFK e z!ndarray_to_mgr..AsT&Sdaqz&Ss)strict) placement)rrr>U)na_value)rE)r8)6rBrr#rnamerIreindexrArCemptyobjectgetattrrrDrErangeshaper7rVr'rNr _referencesrrreshape_values _ensure_2drasarray_prep_ndarrayliker! _get_axes!_check_values_indices_shape_matchTrlistr maybe_convert_objectsanyzip enumerater.r,r+slicekindrrnanr-construct_array_type_from_sequencer/)valuesrIrHr7rV infer_objectvdtyper>nrJ obj_columnsmaybe_datetimedval block_valuesbpnbirWs rMndarray_to_mgrrs "&9e^*LMML&)$ ?{{& . =LLE^^E*F6{w2s7|XXfF3F VWd +F D6"&9%&@ frzz>: ; av||A//Aq!t /  XF     Aa.%!CD'0'1E&:T"  ?E#f+./G"7+GVWe5dSS&9e,-.u*M%%D&.)vT: [[]F ;;! ^^B*F FY. / ^^((*F^^FF# FRZZ8 9 U]nV\\5&IXXf4s;FZZ'FF# #65 V\\U2     QawNE7&feW= XXF  }/&,,*G6l !  !  % %!&$(*/!#'!2 !    Tc+~d&ST T )88GAt/a8NSTDUV8   c'l 34Bf>B4L 6<<,,38J8LRVV,6l %[1  24 **,;;D;NuQA/  2  E#g,/ 0 &BT :t  7|q +w& CV   s U5/A+U:>4U? *VA V c0|jdt|k7s|jdt|k7r`|jddcxk(rt|kr td|j}t|t|f}td|d|y)z\ Check that the shape implied by our axes matches the actual shape of the data. r=rz)Empty data passed with indices specified.zShape of passed values is z, indices imply N)r{rArF)rrIrHpassedimplieds rMrres||A#g,&&,,q/SZ*G <<?a ,#e* ,HI I -u:s7|,5fX=MgYWXX+Hr\r7rVc@|t|}|4t|tjs|jgt |z}ntj gt |z}t}t|j}t|j}t|D]J\} } |j| } || } | || <t| s.t!| s:|j| L|;|r-t#t|D cgc] \} } | |vs | c} } }nt#|}n t|}|rt%|s|D]7} t'|| t |||ntjd} | || <9n_t)t|j}|r t+|n t-d}|Dcgc]}t/j0||}}|rv|Dcgc]k}t|t2r|j5nHt|t*s%t|t6r't9|jr|j5dn|m}}t;|||||S#t$r|j| YwxYwcc} } wcc}wcc}w)z Segregate Series based on type and coerce into matrices. Needs to handle a lot of exceptional cases. Used in DataFrame.__init__ rxrTrQr^)r'rBrCr7rtrArsetkeysrrrget_locKeyErroraddrrr?rr r)r#r&commaybe_iterable_to_listrrVrrrN)rWrIrHr7rVrGmidxs data_keys data_valuesrcolumnidxrrKrkrJs rM dict_to_mgrrwsq w'  Zrxx%@nn%G 4FffXG ,F - 4;;=) "7+IAv ''/ $EF1IDK ! , =&+4V+<O+t|dk(rtjdtSt |t r t |}|dtjfSd}t|dr4tj|Dcgc] }|| c}}t|St |dtjrF|djdk(r4tj|Dcgc] }|| c}}t|S||}t|Scc}wcc}w)Nr)rrrP.cpt|rt|tr|St|d}t |}|S)NTr;)rrBrrr)vress rMconvertz"_prep_ndarraylike..converts4A*Q "=H !4 0$Q' r\) rArCrwrxrBrzr newaxisrrrDrEr)rrVrKrrs rMrrs 6{axxf-- FE "v&3 ?##F1Iv6v!71:v67 f  F1Irzz *vay~~/Bv6v!71:v67 f  f 77s >D!Dc|jdk(r"|j|jddf}|S|jdk7rtd|j|S)zB Reshape 1D values, raise on anything else other than 2D. r=rrqzMust pass 2-d input. shape=)rEr}r{rF)rs rMrrs\{{aa! 45 M  6v||nEFF Mr\c>d}g}g}|D]}t|ttfrj||j|}t|tr|j|ur|j |}|j |j|j}nt|tr||jd}t|ttfr t|}n t|}t|ts4|jr(ddlm}||j |j}n0t#j$||jt&j(}t+|||d}t-j.|||j d|j |||fS)NOr)Series)defaultFr)rBrr#astyperIrvappendr|r~dictr"r%r r$hasnanspandasrr fast_multigetrCrr!rrequire_length_match)rWrIr7oindex homogenizedr>valrs rMr@r@)sWFKD cIu- . jj'#y)ciiu.Dkk%( KK (++C#t$>"\\#.Fem^%DE%c*Cs)C!%4. +--e4<.sKd640w' '>r\c8ddlm}tt||S)a Converts a list of dataclass instances to a list of dictionaries. Parameters ---------- data : List[Type[dataclass]] Returns -------- list_dict : List[dict] Examples -------- >>> from dataclasses import dataclass >>> @dataclass ... class Point: ... x: int ... y: int >>> dataclasses_to_dicts([Point(1, 2), Point(2, 3)]) [{'x': 1, 'y': 2}, {'x': 2, 'y': 3}] r)asdict) dataclassesrrmap)rWrs rMdataclasses_to_dictsrs0# FD! ""r\ct|st|tjr|jj rt |jj }|Dcgc]}|| }}t|dk(r/t|D]!\}}|jdk(s|dddf||<#||fSgt gfSt|tjrM|jj 7|t|jj }|Dcgc]}|| }}||fSt|dttfr t|}nst|dtjrt||\}}nFt|dt rt#||\}}n#|Dcgc] }t|}}t|}t%|||\} }| |fScc}wcc}wcc}w)a  Return list of arrays, columns. Returns ------- list[ArrayLike] These will become columns in a DataFrame. Index This will become frame.columns. Notes ----- Ensures that len(result_arrays) == len(result_index). Nrrq)rArBrCrDr7namesr'rrEr#rtuple_list_to_arraysrMapping_list_of_dict_to_arraysr_list_of_series_to_arrays_finalize_columns_and_data) rWrHr7rurGrrKrrJcontents rMrTrTs$ t9 dBJJ 'zz+&tzz'7'78189$t*9t9>#,F"3388q=(+AqD F1I#4w&<### D"** %$***:*:*F ?DJJ,,-G#*+7a$q'7+w$q'D%=)d# DGS[[ ).tW= W DGY '0w? W#''$Qa$'d#1#wFGW G ?: ,(s G GGct|dtrtj|}|Stj|}|S)Nr)rBrr to_object_array_tuplesto_object_array)rWrs rMrr0s@$q'5!,,T2 N%%d+ Nr\c|3|Dcgc]}t|ttfs|}}t|d}i}g}|D]}t |dd}|t t |}t||vr|t|}n|j|x}|t|<t|d} |jtj| |tj|} | |fScc}w)NFrrITr;)rBrrr(ryr&rAidrrrrtake_ndrCvstack) rWrHrJ pass_data indexer_cachealigned_valuesrrIrrrs rMrr;s  $Q1 1y,6O(PQ Q(?+-MN 7D) =!#a&)E e9 %#BuI.G161B1B71K KGmBuI.q5j00ABii'G G )Rs C*C*c\|Dcgc] }t|tur|n t|"}}|Xd|D}td|D }tj||}t |}tj ||}||fStj |t|}||fScc}w)a Convert list of dicts to numpy arrays if `columns` is not passed, column names are inferred from the records - for OrderedDict and dicts, the column names match the key insertion-order from the first record to the last. - For other kinds of dict-likes, the keys are lexically sorted. Parameters ---------- data : iterable collection of records (OrderedDict, dict) columns: iterables or None Returns ------- content : np.ndarray[object, ndim=2] columns : Index c3NK|]}t|jywrk)rr)rmrJs rMroz*_list_of_dict_to_arrays..vs,t!tAFFH~ts#%c3<K|]}t|tywrk)rBr)rmds rMroz*_list_of_dict_to_arrays..ws9Dqz!T*Dr)typerrr fast_unique_multiple_list_genr'dicts_to_arrayr)rWrHrgenrpre_colsrs rMrrZs28<  W  'o3&'s A A0 A++A0c|tt|}|St|txrt d|D}|s:t|t|k7r#t t|dt|d|rrt|Dchc] }t|c}dkDr t d|r@t|dt|k7r&t t|ddt|d|Scc}w)a If columns is None, make numbers as column names; Otherwise, validate that columns have valid length. Parameters ---------- content : list of np.ndarrays columns : Index or None Returns ------- Index If columns is None, assign positional column index value as columns. Raises ------ 1. AssertionError when content is not composed of list of lists, and if length of columns is not equal to length of content. 2. ValueError when content is list of lists, but length of each sub-list is not equal 3. ValueError when content is list of lists, but length of sub-list is not equal to length of content c3<K|]}t|tywrk)rBr)rmcols rMroz0_validate_or_indexify_columns..s7 -4cJsD !Wrz! columns passed, passed data had z columnsr=z.convertsr BHHSM !(72Xj6XK++&*5C}99 -0=0H-33).,02K)+'):C1SYY"((3-5O$/M "+"@"@"B%44S 4J& %#g-*S"**2Myy~~/&s7  E>2002((E(F t# -S%8 r\rl)rr7rrrrKrGs ``` rMr r s-,4l'. .gsgclgF . M/s") rHr#r7DtypeObj | Noner8boolr9rreturnr5)rWznp.rec.recarray | np.ndarrayr7rrVrr r5)r7rrVrr r5)r np.ndarrayrIr#rHr#r None)rWrr7rrVrr r5) rWr2rH Index | NonerIr#r7rr z$tuple[list[ArrayLike], Index, Index])r r)T)rVrr r!)rr!r r!)rIr#r7rr z!tuple[list[ArrayLike], list[Any]])r r#) rGlist[ArrayLike]rYr#rHr#rintr tuple[list[ArrayLike], Index]) rr%rr%rIr#rHr#r ztuple[Index, Index]rk)rHr#r7rr r&)rWzlist[tuple | list]r r!)rWrrHr#r tuple[np.ndarray, Index])rWz list[dict]rHr#r r')rr!rHr#r7rr r&)rzlist[np.ndarray]rHr#r r#)rF) rzlist[npt.NDArray[np.object_]]r7rrstrrrr r$)c__doc__ __future__r collectionsrtypingrrrrCrpandas._configr pandas._libsr pandas.core.dtypes.astyper pandas.core.dtypes.castr r r rpandas.core.dtypes.commonrrrrrrpandas.core.dtypes.dtypesrrpandas.core.dtypes.genericrrpandas.core.dtypes.missingr pandas.corerrrpandas.core.arraysrpandas.core.arrays.string_rpandas.core.constructionrrrr r!pandas.core.indexes.apir"r#r$r%r&r'r(r)r*pandas.core.internals.blocksr+r,r-r.pandas.core.internals.managersr/r0collections.abcr1r2pandas._typingr3r4r5r6rNr[rrrrrrrr@r?rUrrrrTrrrrrr rlr\rMr>sF # -4 ,.2      "!8 8  8  88 8v &      La#2a:>a aHY Y$Y/4Y Y." UP UP  UP  UP UPp" " " "  " * "4 #L11.1&1h7t */:FPS"6$ '2>$#D;?66(76"6r  >' ' ''T   # ,4 4(44 4t! N *N NN N  Nr\