How to use the elasticsearch2.helpers.scan function in elasticsearch2

To help you get started, we’ve selected a few elasticsearch2 examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github CenterForOpenScience / osf.io / website / search_migration / migrate.py View on Github external
def migrate_collected_metadata(index, delete):
    cgms = CollectionSubmission.objects.filter(
        collection__provider__isnull=False,
        collection__is_public=True,
        collection__deleted__isnull=True,
        collection__is_bookmark_collection=False)

    docs = helpers.scan(es_client(), query={
        'query': {'match': {'_type': 'collectionSubmission'}}
    }, index=index)

    actions = ({
        '_op_type': 'delete',
        '_index': index,
        '_id': doc['_source']['id'],
        '_type': 'collectionSubmission',
        'doc': doc['_source'],
        'doc_as_upsert': True,
    } for doc in list(docs))

    bulk_update_cgm(None, actions=actions, op='delete', index=index)

    bulk_update_collected_metadata(cgms, index=index)
    logger.info('{} collection submissions migrated'.format(cgms.count()))