View all icrawler analysis

How to use the icrawler.utils.Signal function in icrawler

To help you get started, we’ve selected a few icrawler examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

hellock / icrawler / icrawler / crawler.py View on Github

def init_signal(self):
        """Init signal

        3 signals are added: ``feeder_exited``, ``parser_exited`` and
        ``reach_max_num``.
        """
        self.signal = Signal()
        self.signal.set(
            feeder_exited=False, parser_exited=False, reach_max_num=False)