Hj-jUddlmZddlmZmZddlmZddlmZm Z ddl Z ddl m Z ddl Z ddlZddlZddlZddlZddlZddlmZdZd Zd Zgd Zd ed <e j8dZedGddZdddZ d d dZ d!dZ!d"dZ"d#dZ#d$dZ$d%dZ%d&d'dZ& d(dZ' d)dZ(y)*) annotations)ThreadPoolExecutor as_completed) dataclass)date timedeltaN)Path)Settingsz4https://datacenter-web.eastmoney.com/api/data/v1/getz7https://np-anotice-stock.eastmoney.com/api/security/ann)) AUDIT_ADVERSEhardu无法表示意见|否定意见)AUDIT_QUALIFIEDwarninguH保留意见|非标准审计意见|非标审计意见|非无保留意见)REGULATORY_MAJORr uL立案告知书|立案调查|行政处罚决定书|公开谴责|纪律处分)REGULATORY_WARNINGru3警示函|采取监管措施|责令改正|监管函)FUNDS_OCCUPANCYr uK非经营性资金占用|违规资金占用|资金占用事项|违规担保)MAJOR_LITIGATION_DEBTru@重大诉讼|重大仲裁|债务逾期|申请重整|破产重整)DELISTING_RISKr uA可能被终止上市|重大违法强制退市|退市风险警示zlist[tuple[str, str, str]] RISK_PATTERNSuY影响消除|事项消除|清偿完毕|整改完成|撤销风险警示|解除风险警示T)frozenc@eZdZUded<ded<ded<ded<ded<y ) GovernanceBuildResultint pledge_rowsannouncement_rows risk_flagsbool refreshedr output_pathN)__name__ __module__ __qualname____annotations__9C:\FundamentalWorkbench\src\fundamental_mvp\governance.pyrr#sOOr%rc|dztjj|z}tjj |ddd}d}t |D]R} tjj |d5}tj|jcdddcStd |dd |#1swYnxYwt#t$r'}|}tjd|d zzYd}~d}~wwxYw) N?z ,'''<zz(--/2=<" -k$3.?-@A B R =<< ,J JJsgk* + + ,s0!C <#B> C >C C  C;C66C;c(dfd }g}t|5}td|dzDcic]}|j|||} }t| D]!} |j | j # ddd|Scc}w#1swY|SxYw)NcXt}||d|vrdnd<t|}|S)N pageNumber page_index)dictrD)pager=payload base_params data_getterr<s r&fetchz_fetch_pages..fetch?s7k"KO|v5|<HC(7##r%) max_workersr )rJrreturnlist[dict[str, object]])rr3submitrextendresult) r<rL total_pagesrMworkersrNrowsexecutorrJfuturesfutures `` ` r& _fetch_pagesr[=s$ %'D  0HBG;YZ?B[\B[$8??5$/5B[\"7+F KK  (, 1 K] 1 KsBB1BBBct|jd}|jdrd|zS|jdrd|zSy)N6sh)03sz)strzfill startswith)codes r& _source_codergMsC t9??1 D sd{ z"d{ r%c ddddddddd}tt|}t|dd d ddd }d dd dddddd|dd }tt|}t|dd}t t||dd}t j |}|djt|d<|jdddddddd gd!}t j|d|d<|jdg"jd|fS)#N TRADE_DATEz-1r RPT_CSDC_STATISTICSALLWEB) sortColumns sortTypespageSizerG reportNamecolumnssourceclientrTdatar PLEDGE_RATIOi RPT_CSDC_LISTz (TRADE_DATE='z')) rmrnrorGrprqrrrsfilterpagesc|ddS)NrTrtr$xs r&z_fetch_pledge..dsQx[QWEXr%rV SECURITY_CODE source_coderf source_name pledge_datepledge_ratio_pctpledged_shares_10kpledge_deal_countpledge_market_value_10k)rSECURITY_NAME_ABBRrirvREPURCHASE_BALANCEPLEDGE_DEAL_NUMPLEDGE_MARKET_CAPrq)rrfrrrrrr)subset) rDEASTMONEY_DATA_APIrcrr[pd DataFramemaprgrename to_datetimedropnadrop_duplicates) profile_paramsprofilelatestr=firstryrWframeselecteds r& _fetch_pledgerVsJ#$AUV+Z_N*N;G "6*1-l; z&_extract_notice_row..ss/`uUYYz25N1O1Z1Z[^1_Us/99 stock_coder short_nameart_code notice_datetitlez*https://data.eastmoney.com/notices/detail//z.html)rrfrannouncement_idannouncement_daterr<)rnextrgrc)itemrrrs r&_extract_notice_rowrqs HHW  #E``bfgH s8<< b#ABCK" \*||L188J/!XXm4'";HLL.saPViX^N_r%rrrF) ascending) isoformatrDEASTMONEY_NOTICE_APIrr[rrrrr sort_values) startendr=rrrUitemsrrowrWrs r&_fetch_risk_noticesrss!W\q0As}}F *F 3EU6]<01J?s*K -v{D_ij kE! TED-@-F&Fc%SCED T LL  . ./@ AE!#6I0J!KE    13DERWY^Q_  `` Us +C>Cc g}|jdD]}t|j}tj |r.t D]\}}}t j||s|j|jjdrdnd|jddz|j||||j|j|jdtj|gdjgd S) NFindexr_zSH.zSZ.) security_idr rule_codeseverity description event_dater source_urlr)rrr) itertuplesrcrRESOLUTION_PATTERNsearchrreappendrrerrr<rrr)noticesrWrr risk_typerpatterns r&_classify_noticesrs$&D!!!.CII  $ $U + ,9 (Ixyy%( -0__-G-G-MESX\_\k\klmln\o#o#&??X`#(8M8M'*':':#'' -: / <<' FGHr%c :|jjdd|jjdd|jdz }|jdz }|jdz }|jdz }|jxr5t j|j j z |dzk}|r|s|jr|jry|jritj|}tj|} tj|} tt|t| t| d|St j} ttjjd } t!\}} tjt#d z }t%|tj} t'| } |j)|d | j)|d | j+|dd | j-| t||jtjjt| t| dt/t j| z dgdd |j1t3j4| dddt7||| | t9|| | |tt|t| t| d|S)NT)parentsexist_okzexternal_pledge_latest.parquetzexternal_risk_notices.parquetz"external_governance_risk_flags.csvzgovernance_metadata.jsoniQF)schema_version build_datei)daysrz utf-8-sig)rencodingzEastmoney public data APIsr)uK公告风险依据标题分类，需对高风险样本阅读原文复核。u`只拉取近两年风险提示类公告，不能代替交易所和证监会完整监管档案。uE未出现风险标题不等于已确认标准无保留审计意见。) rr notice_start notice_endrisk_notice_rowsclassified_risk_flagsrrelapsed_seconds limitations) ensure_asciiindentutf-8r) processed_dirmkdir report_direxistsr9statst_mtimer read_parquetread_csvrlenGOVERNANCE_SCHEMA_VERSIONrtodayrrrrr to_parquetto_csvupdateround write_textr5dumps_persist_governance _write_report)settingsforce cache_days pledge_path notices_path flags_path metadata_pathfreshpledgerflagsstartedstatusrrs r&refresh_governance_datars}    = dT:((+KKK)),KKL''*NNJ''*DDM  " i m6H6H6J6S6S(SWadiWi'iE U{113 8K8K8MR\RcRcRe-//,/ J'$S[#g,E ES]^^iikG3L\`\f\f\h\r\r\t uF'/FK JJL9#. .E!%6G g &E k/ |51 LL5;L? MM"3v;)9O9O9QL3u:.5W^I^`aCb  TZZU1MX_`&'59(FE62 Vc'lCJj YYr%cVtjt|jdz 5}|j d||j d||j d||j d|j d|j ddddy#1swYyxYw)Nzfundamental_mvp.duckdb pledge_df notices_dfflags_dfzICREATE OR REPLACE TABLE external_pledge_latest AS SELECT * FROM pledge_dfzICREATE OR REPLACE TABLE external_risk_notices AS SELECT * FROM notices_dfzPCREATE OR REPLACE TABLE external_governance_risk_flags AS SELECT * FROM flags_df)duckdbconnectrc database_dirregisterexecute)rr rr  connections r&rrs H114LLM NR\K0L'2J.fgfgmn O N Ns A*BB(c V|jddgjjd}djd|j dD}d |d d |d d t |d|ddt |d|d }|j dz j|dy)Nrrcount)name c3pK|].}d|jd|jd|jd0yw)z| z | z |N)rrr)rrs r&rz _write_report..s4nOm!1;;-s1::,c!''"EOms46Fru7# M1 治理与外部风险数据报告 生成日期：ru 质押数据日期：ru 有质押记录公司：u% 家 近两年风险提示公告：ru 条 分类风险标记：u; 条 | 风险类型 | 级别 | 记录数 | |---|---|---:| u* ## 口径限制 - 公告分类依据公告标题，硬风险样本必须点击保存的原文链接复核。 - 无风险标记只表示近两年未匹配已建规则，不表示公司已被证明没有治理风险。 - 质押数据是当前快照，后续回测必须保存每期历史快照。 u!M1_治理与外部风险报告.mdrr)groupbysize reset_indexjoinrrrr)rmetadatar r countslinesreports r&rrs ]]K4 5 : : < H Hg H VF IInvO`O`glO`Omn nE&'(}-./V &'(:;<=%j\"F$>>JJ6\cJdr%)r)r<rcr=dict[str, object]r>rrPr()r) r<rcrLr(rUrrVrrPrQ)rfrcrPz str | None)rPztuple[pd.DataFrame, str])rr(rPzdict[str, object] | None)rrrrrP pd.DataFrame)rr)rPr))Fr )rr rrrrrPr) rr r r)rr)r r)rPNone) rr r$r(r r)r r)rPr*)) __future__rconcurrent.futuresrr dataclassesrdatetimerrr5pathlibr rr9 urllib.parser.urllib.requestrpandasrconfigr rrrrr#compilerrrDr[rgrrrrrrrr$r%r&r5s"?!$   KP- ) RZZ |} $S$./ '* 3J  Z6& aH(&ZRo+o04oe&e+/er%