'eeddlmZddlmZddlmZddlmZmZmZddlZddl Z ddl m Z ddl Z ddl Z ddlmZddlZddlmZmZmZmZmZdd lmZdd lmZmZmZdd lmZddlZdd l m!Z!m"Z"dd l#m$Z$ddl%m&Z&m'Z'ddl(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0m1Z1ddl2m3Z3ddl4m5Z5ddl6m7Z7ddl8m9cm:Z:ddl;mZ?dZ@d#dZAdZBe!jCfdZDej9jEjFjGd$dZHd%d!ZId"ZJdS)&)print_function)absolute_import)division)rangemapfilterN) defaultdict)datetime)ListOptionalUnionTupleDict)cpapi) MySQLGovernorMySQLGovExceptionGovernorStatus) setup_logging)dateutilfunc)g_usersManager)get_users_for_resellerdeserialize_lve_id)getuser)HistoryShowUnion)HistoryShowDBGov)uid_to_username_username_to_uid_localreseller_to_uid)LVEList)setup_statistics_parser)make_db_engine)main_zN/Ac eZdZ d!dZedZd Zd Zd Zd Z d Z dZ dZ dZ dZdZedZdZdZedZedZdZdZdZdZd"dZedZdZd ZdS)#CloudLinuxStatisticsN?r<c d|_d|_||_||_d|_d|_| |_| |_|jdup t|jttf|_ g|_ g|_ g|_d|vrgd}d|D|_|||_||_|||jg|_g|_d|_d|_||jr| |_n| |_| |_d|_d|_||||_||_t?|_ tCj"|j#j$|_%|&|_'tQdkr|)n tU|_+dS)z Initializes statistics object used by SPA UI. Note, that if we set order_by to mysql_*, and set by_usage or by_fault, end result will will not be ordered by mysql_ column, using uid for ordering instead Nrall) cpuepvmempmemnprocioiopsr0mysqlc6g|]}|S)lower).0ss /builddir/build/BUILDROOT/lve-stats-4.2.1-1.el7.x86_64/opt/cloudlinux/venv/lib/python3.11/site-packages/lvestats/lib/cloudlinux_statistics.py z1CloudLinuxStatistics.__init__..Is ===1QWWYY===root), order_by_lve order_by_gov server_idengine limit_gov limit_lveuid time_unit isinstancelisttuple is_many_uids _lve_results _gov_results_resellers_results show_columnsnormalize_by_faultby_faultorder_by init_order_by_show_columns_lve_show_columns_gov_gov_key_index_lve_key_indexinit_show_columnslimit by_usage_lve by_usage_gov init_by_usageby_usage_percentage thresholdrmysql_governorlogging getLogger __class____name__log _show_mysql show_mysqlr_get_admin_uidsset _admins_uids) selfr@rLrOr?by_usagerZrNr[rVrCrDs r9__init__zCloudLinuxStatistics.__init__(s$! " " H,S 48dE]0S0S"$ L aaaL== ===//99    8T]333!#!#""      #"DNN"DN    8$$$#6 "+oo$T^%<==**,,6=ii66I6ID00222suur;c tdtjDdhz S#tjtf$rtcYSwxYw)Nc34K|]}t|VdSN)r)r7admins r9 z7CloudLinuxStatistics._get_admin_uids..es+QQ-e44QQQQQQr;)reradmins NotSupportedAttributeErrorr5r;r9rdz$CloudLinuxStatistics._get_admin_uidsbsc QQ%,..QQQQQUYTZZ Z"N3   55LLL s-0$AAcF|jod|jvS)Nr3)r\is_governor_presentrL)rgs r9rbz CloudLinuxStatistics._show_mysqlis$"6688YWHY=YYr;c8||vr||SdSrl)index)rglist_keys r9 _index_ofzCloudLinuxStatistics._index_ofms$ %<<;;s## # )rgrOrNs r9rPz"CloudLinuxStatistics.init_order_bys  9~~''H  ** 3$,$4$4Y$C$Cc$I!!!$$W-- 3$,S]]^^$<!!!$'(N!!!  9(1+.. &D    ( 8Xa[D   r;cVt||j|j|j}|ptSrl)rr?r@ NOT_AVAILABLE)rgrCusernames r9 get_usernamez!CloudLinuxStatistics.get_usernames&"3 TT(=(r;ctt}|jD]}|dkr d||d<|jrd|dd<d|dd<|S)zS Return default info about user usage (mysql) :rtype: dict r3lver,r1)r dictrLrc)rg usage_fieldrs r9_get_default_user_usagez,CloudLinuxStatistics._get_default_user_usagesl "$'' ' 1 1F  -0 F#E* ? -*-K w '),K g &r;ctt}|jD]}|dkr d||d<|jr.||\}}||dd<||dd<|S)zT Return default info about user limits (mysql) :rtype: dict r3rrr,r1)r rrLrcget_user_limits_safe)rgr limit_fieldr cpu_limitio_limits r9_get_default_user_limitsz-CloudLinuxStatistics._get_default_user_limitss "$'' ' 1 1F  -0 F#E* ? 2"&";";H"E"E Ix*3K w ')1K g &r;c8i}|jD]}|dkrddi||<|S)Nr3rr)rL)rg fault_fieldrs r9_get_default_user_faultsz-CloudLinuxStatistics._get_default_user_faultss9 ' 1 1F  ',aj F#r;c |jr(t|\}}||}n1t|j\}}||j}t |||}|jdgkr||d<|jr|r%| t ||n| t ||tj |dpttj |dptn:| t|t||jzd|S) N)usagelimitsr3r)r|nameF) raise_exc)r|rdomainreseller)rzr{)rHrrrCrrrrLrupdater get_domainr get_resellerintrD)rgikey_ is_resellerritems r9get_itemzCloudLinuxStatistics.get_items   3/55NA{((..HH/99NA{((22H..0000::<<<   ) )!::<sort_by_another_listr= sort_by_keyrstrftimerawarningformatrDrCjoinrLr?r@rOrNrIrJ)rg lve_results gov_results period_from period_tofrom_gov_to_lve_indexesfrom_lve_to_gov_indexesgov_keyslve_keysnot_in_gov_indexesnot_in_lve_indexeslendifffrom_formatted to_formatteds r9 process_dataz!CloudLinuxStatistics.process_datas6:5I5I+Wb5c5c 3!8 k k  2 f{++KK   f334K[ZlmmKK   f334K[ZlmmKK'+'7'7+xYd'e'e $K{##c+&6&66 a<<(112BCCN$--.>??L H  @AGAG $ $ . , #): ; ; $ AADJJ $ $ $     ('r;c,d}d}t|}t|}g}g} t||zD]W} ||kr<|dg||z z| ||dn||kr;| dg||z z|||dn||||krA|||| |||dz }|dz }||||kr7|d| |||dz }"| d||||dz }Y| |fS)Nrr()rrextendappend) rrrrlve_posgov_poslve_lengov_lenrIrJrs r9rz CloudLinuxStatistics.sort_by_key(sk""k""  w())  A'!!##TFGGO$<===##K$9:::G####TFGGO$<===##K$9:::'"hw&777##K$8999##K$89991 1 '"Xg%666##D)))##K$89991 ##D)))##K$89991 \))r;cg}|D]5}||d|||6|D]}||||}|Srl)r)indexesresultsnot_in_indexes_resultsrus r9rz)CloudLinuxStatistics.sort_by_another_listLs} 0 0E}%%%%////# , ,E OOGEN + + + +r;c jrd|D fd|D}jsjrd|D fd|D}fd|D}fd|D}||}dt |D}||}dt |D}||||||||fS) Ncg|] }|d Srr5r7rs r9r:z8CloudLinuxStatistics.process_indexes..]222!222r;c(g|]}|dv |Srr5)r7rgov_uidss r9r:z8CloudLinuxStatistics.process_indexes..^'LLL3Q89K9K39K9K9Kr;cg|] }|d Srr5rs r9r:z8CloudLinuxStatistics.process_indexes..`rr;c(g|]}|dv |Srr5)r7rlve_uidss r9r:z8CloudLinuxStatistics.process_indexes..arr;c*g|]}|jSr5)rSr7rrgs r9r:z8CloudLinuxStatistics.process_indexes..b!@@@qAd)*@@@r;c*g|]}|jSr5)rTrs r9r:z8CloudLinuxStatistics.process_indexes..crr;cg|] \}}|| Srlr5r7irs r9r:z8CloudLinuxStatistics.process_indexes..e"\\\DAqRSR[aR[R[R[r;cg|] \}}|| Srlr5rs r9r:z8CloudLinuxStatistics.process_indexes..grr;)rXrWrN get_indexes enumerate) rgrrrrrrrrrrs ` @@r9rz$CloudLinuxStatistics.process_indexesZs7   M22k222HLLLL+LLLK   M  M22k222HLLLL+LLLK@@@@K@@@@@@@K@@@"&"2"28X"F"F\\I6M,N,N\\\"&"2"28X"F"F\\I6M,N,N\\\&(? k k  23 3r;c$fd|DS)Nc<g|]}|Sr5)rx)r7rC second_uidsrgs r9r:z4CloudLinuxStatistics.get_indexes..os'GGGS{C00GGGr;r5)rg first_uidsrs` `r9rz CloudLinuxStatistics.get_indexesms!GGGGGJGGGGr;c|rL||j||j}|||j|S|jr ||jkr|jsL||j||j}|||j|S|js|jr|j |}|7|||j }| ||n+||j||j}|||j|S|j|}|7|||j}|||n+||j ||j }| ||j |St)zS :type resellers: bool :type key: int :rtype: dict ) rrKrTrrVrcrIr>rArJrSr IndexError)rgrw resellersrrs r9 get_valuezCloudLinuxStatistics.get_valueqs  ==!8!=d>Q!RSSD%%dD,CC,HII I sTZ//? H}}T%6s%;D? " # #r;c|r@|dr|tdd|_dSd|zf|_dSdS)Nr3rr})rrrXrW)rgrhs r9rYz"CloudLinuxStatistics.init_by_usages[  4""7++ 4$,S]]^^$<!!!$'(NO!!!  4 4r;cZtj|}tj|}|jr|t|j|||j|j|j|j|j|j |j |j |j |j  }||}nt!}|jr[t%|j|||j|j|j|j|j|j |jd|j }nt!}|jrbg}|D][} t3| |j\} } | |jvr)| r|j| F|| \|}|||||dS)zGenerate statistics.) dbenginerrrCrDrLr?rOrhrZrNr[rVT) rrrrDr?rLrOrhrZrVreverserCN)r local_to_gmrQrr@rCrDr?r=rWrZrNr[rBset_normalised_outputproceedrFrcrrRr>rXrAhistory_dbgov_showrHrrTrfrKrr) rgrrperiod_from_gm period_to_gm history_showrrusersrrCrs r9generate_statisticsz(CloudLinuxStatistics.generate_statisticss"-k::+I66  ! !+*lH.!3.**$($<.n & & &L  . . 0 0 0&..00KK&&K ? !**&..!3**$($<nH   ! "" K&&K   E# ' '#5d4;N6O#P#P [$+++ '+2248888LL&&&&K +{KKKKKKr;) r&NNr'Nr(rNr))F)r` __module__ __qualname__ri staticmethodrdrbrxrUrPrrrrrrrrrrrrrr r rrMrYr$r5r;r9r%r%'s  #8U8U8U8Ut\ ZZZ$$$ QQQ4 9 9 9)))      4     \ "("("(H!*!*\!*F  \ 333&HHHB$$$ **** ##\#4445L5L5L5L5Lr;r%c tdkrtjStj||rt |p t}t ttt tt|}t|p t}|r||vs||kr|ngS|| |n|}|S)z= Get uid that should be selected from history table; r<) rosgetuidrbuild_users_cacherrFrboolrrrr) for_resellerrC users_listid_ reseller_uids r9_get_uid_for_selectr1s yyFy{{$\222 +L,EGIIFF 6$S)?%L%L M MNNOO&|'@wyyAA   D**|(;(;33 C  # JJ| $ $ $ Jr;c|jsdSt|j|j}t ||||}t |}|dfS)z Get command response and exitcode. :type log: None | logging.Logger :type engine: sqlalchemy.engine.base.Engine :rtype: (str, int) )z Only JSON mode supported for nowr)jsonr1r-r|get_statistics dump_json)r@optionsrar/ statistics json_datas r9executer: sR <655 g2GJ ? ?CS'::J*%%I a<r;c.tj|}|Srl)r4dumps)r8r9s r9r6r6s :&&I r;c td|}|j4|j-|gkr'd|j|j}||d<|S t ||j|j|j|j|j dz |j |j |j ||j  }|t|d|jt#|}t#|d } |jr ||d <| |d <n||d <nM#t&$r@} t)| |d<|r"|t)| Yd} ~ nd} ~ wwxYwt-j\} } | |d <| t0jkr| j|d<| j|d<|S)Nsuccess)r  timestampz@Permission denied. User with id {} does not belong reseller `{}`r gY@) rLrOr?rhrZrNr[rVrCrDrzT)rr#rusermySqlGovrr)rr-r|rr%showrOr?rh percentagerNr[rVrDr$getattrr{r rH Exceptionstrerrorrget_governor_statusrERRORrr) r@r/rar7r?r  error_msgstatlvelistlvelist_resellersrstatusrGs r9r5r5!sF 'GJ,Bsbyy77=vgj>E>R8T8T %x #  %'% ' 2T 9%'-' ) ) )   &!9!97:FFF$--#DD999   %%F7O"3F;  $F6N q66x   IIc!ff    ,..MFEF: %%%!My!My MsB*C99 E6D>>E localhostc|'tidtjtj}t ||}t |||\}}t ||S)z- Execute command and return exitcode Nr%) caller_name file_level console_levelra)rr]WARNINGFATALr!r:print)r@argv_r?rar7json_str exit_codes r9mainr[Ys_  {B,Bwnun{|||%eY77G!&'s;;;Hi (OOO r;1dcRtj}t|}|dd}dddd|zg}t ||}t ||d\}}t j|}|d } |d } t| t| fS) z: Return amount of users and resellers with faults r?rOz --by-faultr--jsonz --period=NrTrr#) config read_configr"getr!r:r4loadsr) periodcnfrr?opt_listr7rYrr9rr#s r9#get_users_and_resellers_with_faultsrfhs    Cc""H [11IeX{V/CDH%h ::G(G666KHa 8$$I+&I g E u::s9~~ %%r;cvttjddt|dddtt t j|dz dzg}|rtj|} td |d D}n#ttf$rd}YnwxYw|S) Nr^z--idz--show-columnsmPMemz--periodr)m)rXcg|] }|d S)rhr5)r7xs r9r:z*get_max_memory_for_lve..~s???aqz???r;data) r#r_r`rFrmathceilr4rbmax ValueErrorKeyError)lve_idperiod_secondsrl parsed_datamaxPmems r9get_max_memory_for_lvervxs #%%&#f++O_07STYWehjWjMkMkIlIlEmEmpsEs0u v v vD 'j&& ??;v+>???@@  ! NsB B65B6rl)rON)r\)K __future__rrrbuiltinsrrrr]r) collectionsr r4rmr sqlalchemytypingr r r rrclcommonr clcommon.librrrlvestats.lib.commons.logsetuprlvestats.lib.commons.decoratorslvestatslvestats.lib.commonsrr"lvestats.lib.commons.users_managerrlvestats.lib.commons.funcrrclcommon.cpapi.pluginlibrlvestats.lib.lveinfolibrlvestats.lib.lveinfolib_govrlvestats.lib.uidconverterrrrlvestats.lib.lve_listr 1lvestats.lib.parsers.cloudlinux_statistics_parserr!lvestats.lib.dbenginer"lvestats.lib.configlibr_lvestats.lib.info.lveinfomainr#robjectr%r1r:r6gm_datetime_to_unixtimestampr5commons decorators no_sigpiper[rfrvr5r;r9rs&%%%%%&&&&&&'''''''''' ###### 55555555555555IIIIIIIIII777777&&&&////////======PPPPPPPP,,,,,,444444888888^^^^^^^^^^))))))UUUUUU000000$$$$$$$$$////// ELELELELEL6ELELELP4$8(799 5555p  +   ,+  & & & &     r;