So here is my problem: airflow, Since executes a Python file that calls a php script (Symfony) to perform a created command. The command itself works fine. The execution of the latter from Airflow also works very well (with the visual display of Airflow: runs success).
When I cause an error in the symfony command (exit(1), throw,…), to see how Airflow reacts, it always shows success. How do I make him understand that the script didn’t work?
Here is the python code:
dag = DAG( 'name fill python', default_args={ 'start_date': datetime(2022, 8, 1), }, max_active_runs = 1, description='one description', schedule_interval='0 7 * * 1', tags = ["tag1", "tag2"] ) t1 = SimpleHttpOperator( http_conn_id='name fill json', task_id='name task', endpoint='url route symfony', method='GET', data={}, headers={}, dag=dag ) t1
And the route for the Symfony command:
public function ImportMasse(string $csv, KernelInterface $kernel) { $application = new Application($kernel); $application->setAutoExit(false); $input = new ArrayInput([ 'command' => 'Import', 'class_name' => $csv, ]); $output = new BufferedOutput(); $application->run($input, $output); if(stristr($output->fetch(), '0 errors')){ return new Response(true); }else{ return new Response(false); } }
Thanking you for your return.
Advertisement
Answer
The SimpleHttpOperator
allows you to run a check against the requests response object. If the check returns false an AirflowException
would be raise and the operator will fail.
Example:
def check_func(response): # Implement your check logic here if condition_is_ok: return True return False SimpleHttpOperator( ..., response_check=lambda response: True if check_func(response) is True else False )