您正在查看: Surou 发布的文章

推送交易上链“假”失败的处理方案

解决问题

对于通过Rpc推送交易上链,因超时等因素导致无返回交易id或者失败,实际交易已上链。

解决方案

在推送交易时附加一个action,action可以填一个与此次推送任务对应的唯一值mark,并把当前链的块高度记下

void addmark( const name& owner, const string& mark )
{
   require_auth( owner );
}

然后mongo会把这个action数据以及对应的交易Id 洗出来

bool
mongo_db_plugin_impl::add_action_trace( mongocxx::bulk_write& bulk_action_traces, const chain::action_trace& atrace,
                                        const chain::transaction_trace_ptr& t,
                                        bool executed, const std::chrono::milliseconds& now,
                                        bool& write_ttrace )
{
   using namespace bsoncxx::types;
   using bsoncxx::builder::basic::kvp;

   const auto block_time = std::chrono::duration_cast<std::chrono::milliseconds>(
               std::chrono::microseconds{atrace.block_time.to_time_point().time_since_epoch().count()} );

   if( executed && atrace.receipt->receiver == chain::config::system_account_name ) {
      update_account( atrace, now, block_time );
   }
   .....
   else if(atrace.act.name == name("addmark")) {
      add_trx_marks( atrace, now, block_time );
   }
void mongo_db_plugin_impl::add_trx_marks( const chain::action_trace& atrace, const std::chrono::milliseconds& now, const std::chrono::milliseconds& block_time ) {
   using namespace bsoncxx::types;
   using bsoncxx::builder::basic::kvp;
   using bsoncxx::builder::basic::make_document;

   auto addma = fc::raw::unpack<mongo_db_plugin::addmark>(atrace.act.data);

   auto doc = make_document( 
                           kvp( "mark", addma.mark),
                           kvp( "trx_id", atrace.trx_id.str()),
                           kvp( "owner", addma.owner.to_string()),
                           kvp( "contract",atrace.receipt.receiver.to_string()),
                           kvp( "block_time", b_date{block_time}),
                           kvp( "block_timestamp", block_time.count()),
                           kvp( "createdAt", b_date{now} ));

   try {
      if( !_trx_marks.insert_one(doc.view())) {
         EOS_ASSERT( false, chain::mongo_db_insert_fail, "Failed to insert _trx_marks ${name} : ${mark}", ("name", addma.owner.to_string())("mark", addma.mark) );
      }
   } catch (...) {
      handle_mongo_exception( "_trx_marks", __LINE__ );
   }
}

测试查询记录

也就是如果推送没返回交易id,可以尝试查洗出的这个表,如果有的话,使用对应的交易id做数据补全。

{
    "_id" : ObjectId("5df34b721e722d468e02260f"),
    "mark" : "944c1698dc62c1817cc02c68207c47bf85dfc7e830839d2ab7267899bb04ada9",
    "trx_id" : "0e65b2b921019604b7928fdea11fe4e02e10550fe50fe767cbb84043b245e5ee",
    "owner" : "bcskillsurou",
    "contract" : "eosio.token",
    "block_time" : ISODate("2019-12-13T08:27:30.000Z"),
    "block_timestamp" : NumberLong(1576225650000),
    "createdAt" : ISODate("2019-12-13T08:27:30.028Z")
}

如果超过一段时间,且判断mongo中块高度增长正常,且超过前面记下的块高度一段距离(以防意外的话,就是超过当前最新区块与不可逆块的差)【防止mongo同步问题】,也没记录的话,就可以判定失败了。

mongo 同步节点出错,导致数据脏后得处理

讨论前提

方案讨论的前提是在,本地有一份低于mongo数据库块高度得链数据备份
如果没有得话,此文章无效,只能清数据库,整体重放了。所以体现了,定期备份得重要性!

方案实施

查询mongo 数据,确认最后写入得块高度
blocks表,查看最后插入记录得 block_num 块号。
然后查询下其余各表

  • block_states
  • action_traces
  • transaction_traces

将该块号下得所有记录按照交易id,块号都删掉。
因为不能保证最后一个块得数据是写入完整得。
其他的表,插入时有更新判断,不会造成重复,所以无需删除,如果有自行新加得表,自行处理。

启动同步节点

使用之前得低于mongo块高度得链数据备份,启动同步节点
添加--mongodb-block-start参数,指定从哪个块开始,才解析块数据写入mongo.
假设此时mongo删去最后一块后,块高度为1000,那执行开始位置为 1000 + 1

nodeos --mongodb-block-start 1001

开始启动后,节点开始同步,等到1001块时,开始mongo插件开始解析块数据,并写入mongo

附加

由于mongo插件缺少判断,导致提前插入数据,修改如下
https://github.com/EOSIO/eos/blob/3c553db73864ea19458512b4669cf9942dc59f57/plugins/mongo_db_plugin/mongo_db_plugin.cpp#L705

void mongo_db_plugin_impl::process_applied_transaction( const chain::transaction_trace_ptr& t ) {
   try {
      // always call since we need to capture setabi on accounts even if not storing transaction traces
      if( start_block_reached ) { // 新加判断
           _process_applied_transaction( t );
       }
   } catch (fc::exception& e) {
      elog("FC Exception while processing applied transaction trace: ${e}", ("e", e.to_detail_string()));
   } catch (std::exception& e) {
      elog("STD Exception while processing applied transaction trace: ${e}", ("e", e.what()));
   } catch (...) {
      elog("Unknown exception while processing applied transaction trace");
   }
}

获取更全的nodeos日志信息

新建logging.json日志配置文件

{
    "includes": [],
    "appenders": [{
        "name": "stderr",
        "type": "console",
        "args": {
            "stream": "std_error",
            "level_colors": [{
                "level": "debug",
                "color": "green"
            }, {
                "level": "warn",
                "color": "brown"
            }, {
                "level": "error",
                "color": "red"
            }]
        },
        "enabled": true
    }, {
        "name": "stdout",
        "type": "console",
        "args": {
            "stream": "std_out",
            "level_colors": [{
                "level": "debug",
                "color": "green"
            }, {
                "level": "warn",
                "color": "brown"
            }, {
                "level": "error",
                "color": "red"
            }]
        },
        "enabled": true
    }],
    "loggers": [{
        "name": "default",
        "level": "info",
        "enabled": true,
        "additivity": false,
        "appenders": ["stderr"]
    }, {
        "name": "net_plugin_impl",
        "level": "debug",
        "enabled": true,
        "additivity": false,
        "appenders": ["stderr"]
    }]
}

并将其放在config所在目录,重新运行nodeos,即可在终端看到更详细的日志。

通过RPC推送交易后的返回数据,只为参考方便提取相关字段

const result = await api.transact({
        actions: [{
            account: "eosio.token",
            name: 'transfer',
            authorization: [
            {
                actor: "bcskillsell",
                permission: 'active',
            }],
            data: {
                from:"bcskillsell",
                to:"bcskillbuy",
                quantity: '12.00000000 EOS',
                memo: 'bcskill.com'
            },
        }]
      }, {
        blocksBehind: 3,
        expireSeconds: 30,
      });
console.log('\n\nTransaction pushed!\n\n' + JSON.stringify(result, null, 2));
{
  "transaction_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
  "processed": {
    "id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
    "block_num": 32815165,
    "block_time": "2019-12-03T08:28:53.500",
    "producer_block_id": null,
    "receipt": {
      "status": "executed",
      "cpu_usage_us": 427,
      "net_usage_words": 18
    },
    "elapsed": 427,
    "net_usage": 144,
    "scheduled": false,
    "action_traces": [
      {
        "receipt": {
          "receiver": "eosio.token",
          "act_digest": "ce66f1905d2afbc792deaeca80a4e40576e25f3f8f3c204ac7d6ba4aef1fe437",
          "global_sequence": 36674138,
          "recv_sequence": 279646,
          "auth_sequence": [
            [
              "bcskillsell",
              205
            ]
          ],
          "code_sequence": 1,
          "abi_sequence": 1
        },
        "act": {
          "account": "eosio.token",
          "name": "transfer",
          "authorization": [
            {
              "actor": "bcskillsell",
              "permission": "active"
            }
          ],
          "data": {
            "from": "bcskillsell",
            "to": "bcskillbuy",
            "quantity": "12.00000000 EOS",
            "memo": "bcskill.com"
          },
          "hex_data": "00008851e1ae125e0000005e9fae125e008c8647000000000846534300000000137b227472616e736665725f74797065223a337d"
        },
        "context_free": false,
        "elapsed": 166,
        "console": "",
        "trx_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
        "block_num": 32815165,
        "block_time": "2019-12-03T08:28:53.500",
        "producer_block_id": null,
        "account_ram_deltas": [],
        "except": null,
        "inline_traces": [
          {
            "receipt": {
              "receiver": "bcskillsell",
              "act_digest": "ce66f1905d2afbc792deaeca80a4e40576e25f3f8f3c204ac7d6ba4aef1fe437",
              "global_sequence": 36674139,
              "recv_sequence": 122,
              "auth_sequence": [
                [
                  "bcskillsell",
                  206
                ]
              ],
              "code_sequence": 1,
              "abi_sequence": 1
            },
            "act": {
              "account": "eosio.token",
              "name": "transfer",
              "authorization": [
                {
                  "actor": "bcskillsell",
                  "permission": "active"
                }
              ],
              "data": {
                "from": "bcskillsell",
                "to": "bcskillbuy",
                "quantity": "12.00000000 EOS",
                "memo": "bcskill.com"
              },
              "hex_data": "00008851e1ae125e0000005e9fae125e008c8647000000000846534300000000137b227472616e736665725f74797065223a337d"
            },
            "context_free": false,
            "elapsed": 5,
            "console": "",
            "trx_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
            "block_num": 32815165,
            "block_time": "2019-12-03T08:28:53.500",
            "producer_block_id": null,
            "account_ram_deltas": [],
            "except": null,
            "inline_traces": []
          },
          {
            "receipt": {
              "receiver": "bcskillbuy",
              "act_digest": "ce66f1905d2afbc792deaeca80a4e40576e25f3f8f3c204ac7d6ba4aef1fe437",
              "global_sequence": 36674140,
              "recv_sequence": 906,
              "auth_sequence": [
                [
                  "bcskillsell",
                  207
                ]
              ],
              "code_sequence": 1,
              "abi_sequence": 1
            },
            "act": {
              "account": "eosio.token",
              "name": "transfer",
              "authorization": [
                {
                  "actor": "bcskillsell",
                  "permission": "active"
                }
              ],
              "data": {
                "from": "bcskillsell",
                "to": "bcskillbuy",
                "quantity": "12.00000000 EOS",
                "memo": "bcskill.com"
              },
              "hex_data": "00008851e1ae125e0000005e9fae125e008c8647000000000846534300000000137b227472616e736665725f74797065223a337d"
            },
            "context_free": false,
            "elapsed": 6,
            "console": "",
            "trx_id": "7c1f2e70f612bbeaaa1c6687b0da21dda85fb30de5d666f15baa092b70d55f2b",
            "block_num": 32815165,
            "block_time": "2019-12-03T08:28:53.500",
            "producer_block_id": null,
            "account_ram_deltas": [],
            "except": null,
            "inline_traces": []
          }
        ]
      }
    ],
    "except": null
  }
}

EOS 整个链停止,重新强制引导启动

BP节点全部停止,各节点重新启动后,无法正常出块

最近因为非技术原因,导致测试链所有BP节点都停止了。
由于producer_plugin中的安全检查,导致BP节点启动后,一直处于start_block_result::waiting
导致整个链网络瘫痪。

强制引导启动

先准备一台非目前BP节点所在的机器做强制引导节点。然后把目前出块账户的 2/3 +1 个BP账号都配置到此节点。
并把目前其他的BP节点都关闭pkill nodeos

修改强制引导节点的链程序,绕过安全检查

https://github.com/EOSIO/eos/blob/3c553db73864ea19458512b4669cf9942dc59f57/plugins/producer_plugin/producer_plugin.cpp#L1350
将下面代码注释掉,并重新编译链程序

if (_pending_block_mode == pending_block_mode::speculating) {
      auto head_block_age = now - chain.head_block_time();
      if (head_block_age > fc::seconds(5))
         return start_block_result::waiting;
   }

修改BP节点的配置为同步节点的配置

先将其余BP节点的config,改为同步节点的配置,也就是把强制引导节点的p2p-peer-address加到各BP节点的config,并把各BP节点的producer-namesignature-provider都注释掉。

强制启动引导节点

添加-e,强制启动出块

nodeos -e

此时强制出块节点应该已出块。

启动各BP节点

将其余BP节点分别启动,此时开始同步区块,等到追上强制出块节点高度。

迁移BP账号

关闭强制出块节点,把BP账号从config里面每注释掉一部分,就把对应的BP节点停止,并修改config,把对应的BP账号放开。此时重新启动强制出块节点和新迁移BP的节点,现在强制出块节点除去注释掉的部分出块账号。其余BP账号出块正常,且能从新迁移BP节点同步对应的出块BP账号的出块。新迁移的BP节点,出块也正常。
依次类推,迁移其余BP节点账号,等其余BP节点都出块正常后,及此次强制引导启动链网络结束。

备注

能出现全部节点停止的情况,基本都是特殊公链-“私链”,所以具有大部分BP的掌控权,我们先只针对此情景提出修复方案。