2
0
mirror of https://github.com/xcat2/xcat-core.git synced 2025-06-21 05:25:32 +00:00
Commit Graph

233 Commits

Author SHA1 Message Date
2304b0ef50 Add regex helpers
a2idx: Turns an ASCII character into an index (a=1, b=2, etc)
a2zidx: Turns an ASCII character into a 0-index (a=0, b=1, etc)
dim2idx: Turns dimensions (eg row, column, rack-unit) into an index
skip: Return an index where some items are 'skipped'
ipadd: Add a value to an IP address
2017-09-10 23:02:06 -04:00
580b399ade Add trace log in db subroutines (#3666)
This patch add trace log for the db access. Currently support
5 log levels:
- 0: disable the trace log for db.
- 1: trace the calls of database subroutines.
- 2: trace the event to build the cache for the table
- 3: trace the event with cache hit
- 4: trace the SQL statement

implement-feature: #3612

Output example from xcat/cluster.log:
···
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::setAttribs"},"type":"end","elapsed":"0.00176s"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::setAttribs"},"type":"start"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"addon":"SELECT * FROM site WHERE \"value\" = ? AND \"comments\" = ? AND \"disable\" = ? AND \"key\" = ?","table":"site","method":"xCAT::Table::setAttribs"},"type":"start_sql"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::setAttribs"},"type":"end_sql","elapsed":"0.00043s"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"addon":"INSERT INTO site (\"value\",\"comments\",\"key\",\"disable\") VALUES (?,?,?,?)","table":"site","method":"xCAT::Table::setAttribs"},"type":"start_sql"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"addon":"INSERT INTO site (\"value\",\"comments\",\"key\",\"disable\") VALUES (?,?,?,?)","table":"site","method":"xCAT::Table::setAttribs"},"type":"end_sql","elapsed":"0.00043s"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::setAttribs"},"type":"end","elapsed":"0.00171s"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::getAttribs"},"type":"start"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"addon":"SELECT * FROM site WHERE \"key\" = ? and (\"disable\" is NULL or \"disable\" in ('0','no','NO','No','nO'))","table":"site","method":"xCAT::Table::getAttribs"},"type":"start_sql"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::getAttribs"},"type":"end_sql","elapsed":"0.00061s"}
Aug  9 01:59:15 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::getAttribs"},"type":"end","elapsed":"0.00125s"}
Aug  9 01:59:19 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"table":"site","method":"xCAT::Table::getAllAttribs"},"type":"start"}
Aug  9 01:59:19 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"addon":"SELECT * FROM site WHERE \"disable\" is NULL or \"disable\" in ('0','no','NO','No','nO')","table":"site","method":"xCAT::Table::getAllAttribs"},"type":"start_sql"}
Aug  9 01:59:19 c910f05c01bc02k74 xcat[694]: [DB Trace]: {"msg":{"addon":"SELECT * FROM site WHERE \"disable\" is NULL or \"disable\" in ('0','no','NO','No','nO')","table":"site","method":"xCAT::Table::getAllAttribs"},"type":"end_sql","elapsed":"0.00080s"}
···
2017-08-14 21:35:33 -05:00
1826f8e75e Merge pull request #3557 from chenglch/table
Retry 3 times if database connection can not be established
2017-08-07 17:03:24 +08:00
8373b5de6b Optimize the performance for getNodesAttribs and getAllNodeAttribs (#3593)
As `get_site_attribute` subroutine could not cache the value for
hierarchicalattrs configuration, this commit move the check logic of
`hierarchicalattrs` from `getNodeAttribs_nosub_returnany` to `getNodesAttribs`
and `getAllNodeAttribs` to optimize the the performance of bulk query.

After testing, this change will bring 50% performance improvement for
`getNodesAttribs` subroutine.
2017-08-02 21:17:53 -05:00
fdaeb5d761 Retry 3 times if database connection can not be established
As connection may fail due to the limit or error from database side,
this patch add retry mechanism to reduce probability of failure.

fix-issue: #2529
2017-07-28 13:53:34 +08:00
b35ce9a148 Fix regex issue that duplicate the node name
close-issue: #2582
2017-03-03 11:06:01 +08:00
98785fbddf Make nics attribute inhertable
As the attribute value in the nics table can not match the regular
expresion format supported by the table library, this patch add the
logic in the def utils to handle the regular expresion.

close-issue: #2412
2017-02-09 14:25:53 +08:00
15eeb3b2d5 Skit auditlog and eventlog table as auto increment key is uesd (#2066)
Close-issue: #2065
2016-11-02 15:20:08 +08:00
331d7868fa Set value to '' if value within composite primary keys not defined (#2045)
As history reasons, null value is always set within the composite primary
keys, this patch is just a work aroud for the issue encountered on postgres.
Correct fix is to report error to the client side, but too much error handler
is missing in xcat code

close-issue: #2037
close-issue: #2007
2016-11-01 10:14:54 +08:00
0a3fbd8d51 perltidy all perl files 2016-07-20 11:40:27 -04:00
0eb9929452 Add -1 value for $dbworkerpid to make the db access mode more clear.
$dbworkerpid = -1 db process has not been started,
               access db in direct access mode.
$dbworkerpid == 0 db process itself.
$dbworkerpid >0 db process is started, access db in cache mode.
2016-05-25 02:02:48 -04:00
6de5c04f10 Fix typo error about hierarchical attributes 2016-05-17 02:08:12 -04:00
20d3453ad0 Do not undef nodelist reference in DB process
This patch add check method to wait for the DB process in xcatd.
Enhance the close method in Table.pm to disable the undef operation
for nodelist in DB process as the reference has already been cached
as a weak reference.

close-issue: #1011
2016-05-05 06:07:01 -04:00
951aa06963 nodelist table object should not fetch the attribs in groups
This patch intent to fix the undef reference for nodelist temporarily.

partial-issue #1011
2016-05-03 22:50:03 -04:00
45b13721d1 Add configuration in site table to inherit attributes
'heirarchicalattrs' is added in site table to apply attributes
from multiple groups in order.

close-issue #115
2016-04-22 04:53:24 -04:00
ee1d71fe2f Partial issue #537, try to detect the error of db access process
This patch add changes to detect the error and enter in direct db access
mode so that the other process can access the data without db access
process temporarily. Direct access is not suggested way and currently
nodelist (noderange) can not support this mode.

Partial issue: #537
2016-01-21 00:35:13 -05:00
9d0b4b1be7 4390: In DB Access process, recover the signal handle in %SIG which was cleaned up by accident. This fix should be removed if fixed the root cause of %SIG clean up. 2014-11-24 09:41:43 -05:00
afe37ebeab initialize rc var in Table.pm and tabutils.pm to remove warnings when running xcatd in foreground 2014-11-19 17:34:04 -05:00
f0b9cac61c defect 4304: fixed several things: 1. how to handle ctrl+c; 2. handle the plugin process cannot be stopped by TERM signal to process group SSL_Listener. (make the plugin process locate in the SSL_listener process group); 3. Make DB Access process exits later than in processing request. 2014-11-06 06:24:50 -05:00
42b5373796 fix bug 2013-08-06 09:43:40 -04:00
38c2e8e06f add MAXMIN routine 2013-08-05 14:17:48 -04:00
d7c4c6e5f9 Fix problem where setting a string that happened to contain the letters 'NULL' somewhere would fail
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@17024 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-07-25 15:14:21 +00:00
0eecf43a7e Fix problem where dbworkersocket rename could indicate a missing path
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@16743 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-06-25 13:28:52 +00:00
e2e793d864 fix defect 3580/3586
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@16401 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-05-22 18:24:28 +00:00
e11519483c remove unused getNodeSpecAttribs routine
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@16396 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-05-22 17:23:42 +00:00
6c19753851 move "dbworker.sock.$$" from "/tmp/xcat" to "/var/run/xcat/" to avoid mis-deletion
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@16288 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-05-14 05:18:56 +00:00
4a30157492 Apply methodology from previous commit to a number of other places
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@15831 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-04-04 20:11:28 +00:00
abf400572b handle easy regx expressions
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@15703 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-03-28 12:06:27 +00:00
ef9100c56e Correct problem where /// regexes corrupt other substitutions
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@15021 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2013-01-30 16:33:02 +00:00
3918845c64 Fix problem where 'delEntries' failed to invalidate cached data
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@14372 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-11-20 14:51:29 +00:00
90930636d6 Fix problem where a Table object with alternate autocommit setting would not have cache expired by peer activity.
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@14370 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-11-20 14:45:16 +00:00
c14747ac04 Table.pm in a refresh scenario with cache not currently in use, but existing none the less, mark the cache as invalid by setting the timestamp to 1970
Still avoid *rebuilding* the cache, as in the use_cache not set suggests a mass amount of setAttribs and build_cache(1) would be extremely expensive,
just force the next consumer after the set to take the hit

git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@14001 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-10-12 16:31:55 +00:00
5c1a1383f9 document getNodeAttribs improvement
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@13627 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-08-28 15:07:28 +00:00
ac30816620 fix perl INC path
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@13450 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-08-08 17:27:50 +00:00
65f12a5022 Wrap unlikely scenario in eval to hopefully keep DB worker up in the face of some nebulous problem
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@13139 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-06-21 15:19:06 +00:00
eaec3c372b Fix problem where nodels could return more data than requested
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12924 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-24 17:14:38 +00:00
c12bf5a549 fix comments for getAttribs
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12664 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-14 13:44:00 +00:00
112389df2b Add more info on delEntries
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12652 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-13 10:36:16 +00:00
00afad1db2 Correct mistaken variable prefix in optimization work
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12651 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-12 12:03:07 +00:00
83be6fd96e Remove _clear_cache, we are instead just going to let things age out and rely upon use_cache to control when a caller
urgently needs to assure fresh data

git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12648 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-12 11:54:49 +00:00
c82eea4b25 When doing getAllNodeAttribs, institute a noderange result cache to avoid many noderange() calls
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12645 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-11 19:33:27 +00:00
ef27e8e45e More aggressively retain cache
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12641 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-11 18:42:47 +00:00
9f7c4f2dec Change option to 'prefetchcache'
Ensure reference counter doesn't go crazy on prefetch

git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12628 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-11 14:14:48 +00:00
b8f7f7a241 give better example for delEntries
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12620 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-11 11:55:08 +00:00
3a8ee9cf5b give better example for delEntries
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12619 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-11 11:40:38 +00:00
996604e435 Add 'eagercache' to getNodeAttribs
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12613 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-10 20:57:08 +00:00
bb760cf069 Try to do more reuse of nodelist objects in db worker
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12608 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-10 19:59:49 +00:00
abd9669f93 Have Table.pm force flush of more than 5 seconds old, in preparation for more aggresive cache retention
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12481 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-05-02 19:57:49 +00:00
5e9caf5a66 Actually fix typo
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12342 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-04-25 18:24:40 +00:00
16843a2df4 Correct typo
git-svn-id: https://svn.code.sf.net/p/xcat/code/xcat-core/trunk@12341 8638fb3e-16cb-4fca-ae20-7b5d299a9bcd
2012-04-25 18:24:32 +00:00