How to use the ramp-engine.ramp_engine.aws.api._training_finished function in ramp-engine

To help you get started, we’ve selected a few ramp-engine examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github paris-saclay-cds / ramp-board / ramp-engine / ramp_engine / aws / worker.py View on Github external
def _is_submission_finished(self):
        return aws._training_finished(
            self.config, self.instance.id, self.submission)
github paris-saclay-cds / ramp-board / ramp-engine / ramp_engine / aws / aws_train.py View on Github external
# start training HERE
                exit_status = launch_train(
                    conf_aws, instance_id, submission_name)
                if exit_status != 0:
                    logger.error(
                        'Cannot start training of submission "{}"'
                        ', an error occured.'.format(label))
                    continue
                set_submission_state(config, submission_id, 'training')
                _run_hook(config, HOOK_START_TRAINING, submission_id)

            elif state == 'training':
                # in any case (successful training or not)
                # download the log
                download_log(conf_aws, instance_id, submission_name)
                if _training_finished(conf_aws, instance_id, submission_name):
                    logger.info(
                        'Training of "{}" finished, checking '
                        'if successful or not...'.format(label))
                    submission = get_submission_by_id(config, submission_id)
                    actual_nb_folds = get_event_nb_folds(
                        config, submission.event.name
                    )
                    if _training_successful(
                            conf_aws,
                            instance_id,
                            submission_name,
                            actual_nb_folds):
                        logger.info('Training of "{}" was successful'
                                    .format(label))
                        if conf_aws.get(MEMORY_PROFILING_FIELD):
                            logger.info('Download max ram usage info of "{}"'