使用Azkaban调度MaxCompute

环境准备

1.下载并安装MaxCompute客户端,参考链接客户端安装

2.下载并安装Azkaban

开发任务

workflow

使用Azkaban调度MaxCompute

start.job

#start

type=command

command=echo 'job start'

upload_data.job

#upload_data

type=command

dependencies=start

command=/root/odps/bin/odpscmd -f 'upload.sql'

upload.sql

drop table if exists azkaban_emp;

create table  azkaban_emp

   (empno bigint,

    ename string,

    job string,

    mgr bigint,

    hiredate datetime,

    sal bigint,

    comm bigint,

    deptno bigint) lifecycle 1;

tunnel upload emp.txt azkaban_emp;

emp.txt

7369,SMITH,CLERK,7902,1980-12-17 00:00:00,800,,20

7499,ALLEN,SALESMAN,7698,1981-02-20 00:00:00,1600,300,30

7521,WARD,SALESMAN,7698,1981-02-22 00:00:00,1250,500,30

7566,JONES,MANAGER,7839,1981-04-02 00:00:00,2975,,20

7654,MARTIN,SALESMAN,7698,1981-09-28 00:00:00,1250,1400,30

7698,BLAKE,MANAGER,7839,1981-05-01 00:00:00,2850,,30

7782,CLARK,MANAGER,7839,1981-06-09 00:00:00,2450,,10

7788,SCOTT,ANALYST,7566,1987-04-19 00:00:00,3000,,20

7839,KING,PRESIDENT,,1981-11-17 00:00:00,5000,,10

7844,TURNER,SALESMAN,7698,1981-09-08 00:00:00,1500,0,30

7876,ADAMS,CLERK,7788,1987-05-23 00:00:00,1100,,20

7900,JAMES,CLERK,7698,1981-12-03 00:00:00,950,,30

7902,FORD,ANALYST,7566,1981-12-03 00:00:00,3000,,20

7934,MILLER,CLERK,7782,1982-01-23 00:00:00,1300,,10

7948,JACCKA,CLERK,7782,1981-04-12 00:00:00,5000,,10

7956,WELAN,CLERK,7649,1982-07-20 00:00:00,2450,,10

7956,TEBAGE,CLERK,7748,1982-12-30 00:00:00,1300,,10

mc.job

#mc.job

type=command

command=/root/odps/bin/odpscmd -f 'cat_data.sql'

dependencies=upload_data

cat_data.sql

select * from azkaban_emp;

将文件压缩成zip包上传

使用Azkaban调度MaxCompute

使用Azkaban调度MaxCompute

执行

使用Azkaban调度MaxCompute


查看结果

使用Azkaban调度MaxCompute

上一篇:使用日志审计查看MaxCompute执行过哪些操作


下一篇:调度参数在MaxCompute的使用