批量数据插入优化

//原代码
$work = $this->work;
$mail_list = explode(',',$work->mail_list);
foreach ($mail_list as $key => $value) {
   if(empty($value)) continue;
   $send_log = new SendLog();
   $send_log->work_id = $work->id;
   $send_log->phone = $value;
   $send_log->send_time =$work->send_time;
   $send_log->user_id = $work->user_id;
   $send_log->sms_number = $work->sms_number;
   $send_log->type = 'market';
   $send_log->create_time = time();
   $send_log->status = 3;
   $send_log->save();
}

源代码是单个插入,foreach一次就插入一次数据,数据量大的时候 $work变量占用内存很大,每次循环$work->id $work->send_time需要占用很长时间,导致进程一直没有关闭

linux top命令  linux 查看php-fpm 进程数   netstat -napo |grep "php-fpm" | wc -l

php-fpm 进程数量已经开到最大,cpu基本跑慢了

$id= $work->id;
$send_time  = $work->send_time;
$user_id    = $work->user_id;
$sms_number = $work->sms_number;
            
$mail_list = explode(',',$work->mail_list);
$i = 0;
var_dump('循环开始'.date("Y-m-d H:i:s"));

foreach ($mail_list as $value) {
           if(empty($value)) continue;
           $insert[$i]['work_id'] = $id;
            $insert[$i]['phone'] = $value;
            $insert[$i]['send_time'] =$send_time;
            $insert[$i]['user_id'] = $user_id;
            $insert[$i]['sms_number'] = $sms_number;
            $insert[$i]['type'] = 'market';
            $insert[$i]['create_time'] = time();
            $insert[$i]['status'] = 3;
                 
              $i++;
}
 //生成批量插入数据
 foreach (array_chunk($insert,1000) as $kk => $vv) {
      var_dump($kk.'时间'.date("Y-m-d H:i:s"));
      SendLog::insert($vv);
}

优化方案

1、$key换成$i 如果只一维数组直接【】

2、foreach开始前,将变量获取到

3、将一个个数据插入改为批量插入

5万数据从第一种的6分钟缩短为优化后的5秒

php-fpm进程从150降到3(4核8G服务器)

上一篇:tomcat各目录的作用


下一篇:# [USACO09OPEN]Work Scheduling 经典反悔贪心 ##