MuleSoft's Scheduler and Poll Scope components provide a structured way to automate flows at predefined intervals, enabling real-time or near-real-time integrations with external systems.
Schedulers define when and how frequently a flow should trigger, while Poll Scopes wrap inbound endpoints to periodically fetch or monitor data from sources like databases, APIs, or files.
Proper configuration ensures optimized resource usage and prevents overwhelming backend systems. Both components allow advanced options like initial delays, time zones, and fixed or dynamic frequency intervals.
Schedulers and Poll Scopes are particularly useful in ETL pipelines, batch processing, and event-driven architectures where predictable and controlled execution is required.
Understanding the trade-offs between Scheduler and Poll Scope, such as thread usage, error handling, and transactional boundaries, is crucial for designing robust and high-performance MuleSoft applications.
A Scheduler in MuleSoft triggers a flow at predefined intervals, independent of incoming events. It is mainly used to initiate flows at regular or scheduled times, such as hourly data processing or nightly batch jobs.
A Poll Scope, on the other hand, wraps an inbound connector or source and periodically polls it for new data. The polling interval is defined within the scope, allowing the flow to act only when new records or messages are available.
While Schedulers are general-purpose timers, Poll Scopes are more integrated with connectors, making them ideal for scenarios where data needs to be retrieved incrementally without missing updates.
Schedulers allow configuration of frequency interval, initial delay, and time zone to control when flows are triggered. Connector type is not a Scheduler configuration but pertains to Poll Scopes or connectors themselves.
These options enable precise scheduling of tasks in different environments and time zones, preventing overlap and resource contention.
When using Poll Scopes in high-throughput applications, it's essential to consider thread usage, batch size, and frequency. Polling too frequently can overload the backend system, while polling too slowly can increase latency.
Error handling and retry policies must be carefully configured to prevent repeated failed attempts from consuming resources. Using non-blocking processing and asynchronous flows can help optimize performance.
Additionally, leveraging database indexes, filtering at the source, and limiting the data retrieved in each poll are practical strategies to maintain scalability and responsiveness in enterprise integrations.
This flow uses a Scheduler with a frequency of 300000 milliseconds (5 minutes) to trigger the flow repeatedly.
The logger demonstrates that the flow is executed each time the Scheduler fires. This is a basic setup suitable for simple recurring tasks.
// XML
<flow name="scheduledFlow">
<scheduler doc:name="Scheduler" frequency="300000"/>
<logger message="Flow triggered by Scheduler" level="INFO"/>
</flow>The Poll Scope wraps a database select operation, polling every 10000 milliseconds (10 seconds).
It retrieves only new orders and logs them. Poll Scopes efficiently fetch incremental data without requiring external triggers.
// XML
<flow name="dbPollFlow">
<poll doc:name="Poll" frequency="10000">
<db:select config-ref="Database_Config" doc:name="Select New Records">
<db:sql>SELECT * FROM orders WHERE status='NEW'</db:sql>
</db:select>
<logger message="Polled record #[payload]" level="INFO"/>
</poll>
</flow>Schedulers do not automatically retry failed executions; errors can propagate depending on configuration.
Poll Scopes support transactional boundaries, ensuring that if a processing error occurs, the data can remain unprocessed and retried.
Errors are not ignored by default; proper error handling must be configured to maintain data integrity.
Schedulers are preferable when you need to execute a flow at a fixed interval regardless of the presence of new data. For example, generating a nightly report or sending periodic system health checks.
Unlike Poll Scopes, which monitor external sources for new events, Schedulers operate independently and are ideal for batch jobs, housekeeping tasks, or recurring notifications where timing is critical.
This allows consistent execution timing without being tied to the availability of new messages or records in a source system.
The Poll Scope triggers an HTTP GET request every 30 seconds. Responses containing 'error' are logged as errors, others as info.
This setup demonstrates using Poll Scope with conditional processing to separate valid and problematic data dynamically.
// XML
<flow name="apiPollFlow">
<poll doc:name="Poll" frequency="30000">
<http:request config-ref="HTTP_Request_Config" method="GET" url="https://api.example.com/data" />
<choice doc:name="Filter Errors">
<when expression="#[payload contains 'error']">
<logger message="Error found in payload: #[payload]" level="ERROR"/>
</when>
<otherwise>
<logger message="Valid payload: #[payload]" level="INFO"/>
</otherwise>
</choice>
</poll>
</flow>High-frequency polling can overload source systems; realistic intervals are essential.
Filtering and batch sizing reduce data load per poll, improving performance.
Asynchronous processing helps maintain responsiveness and prevents bottlenecks in high-volume scenarios.
The flow uses a cron-based Scheduler to run at 2 AM daily. The try-catch structure ensures that errors can be retried using the Until Successful scope up to 3 times with 5-second intervals.
This demonstrates advanced error handling combined with precise scheduling, suitable for critical nightly jobs where reliability is essential.
// XML
<flow name="dailySchedulerFlow">
<scheduler doc:name="Scheduler" cronExpression="0 0 2 * * ?"/>
<try doc:name="Try">
<logger message="Executing nightly task" level="INFO"/>
</try>
<catch-exception-strategy doc:name="Catch">
<until-successful maxRetries="3" millisBetweenRetries="5000">
<logger message="Retrying failed task" level="WARN"/>
</until-successful>
</catch-exception-strategy>
</flow>To ensure that a Poll Scope processes only new records, implement incremental fetching by tracking a unique identifier or timestamp field.
This can be done by storing the last processed value in a persistent object store or database and using it in the WHERE clause of your query or API call.
This approach prevents duplicates and ensures consistent data processing, particularly in batch integrations.
Filtering data at the source reduces the amount of payload processed, which directly lowers CPU usage.
Asynchronous processing allows handling multiple events concurrently without blocking the main thread, optimizing CPU usage.
Increasing batch size may increase CPU load, and using a Scheduler does not inherently solve high-volume processing issues.
The frequency is set to 900000 milliseconds (15 minutes).
This simple flow demonstrates a recurring task using Scheduler for routine operations.
// XML
<flow name="quarterHourlyFlow">
<scheduler doc:name="Scheduler" frequency="900000"/>
<logger message="Scheduler executed" level="INFO"/>
</flow>Poll Scopes can be wrapped in transactional boundaries to ensure that data processing either completes entirely or rolls back if an error occurs.
This prevents partial processing or data loss when consuming messages from sources like databases, JMS queues, or APIs.
Transactional boundaries are crucial in high-volume, sensitive integrations to maintain data consistency and reliability across multiple systems.
Extremely low polling intervals can overwhelm the backend with frequent requests, causing performance degradation.
Mule runtime threads may become saturated, leading to thread starvation and delays in other flows.
New records are unlikely to be missed; low frequency actually polls more often.
The Poll Scope polls the JMS queue 'incomingQueue' every 20 seconds.
Logging allows monitoring of consumed messages. Polling ensures messages are processed incrementally.
// XML
<flow name="jmsPollFlow">
<poll doc:name="Poll" frequency="20000">
<jms:consume config-ref="JMS_Config" destination="incomingQueue" />
<logger message="Received JMS message: #[payload]" level="INFO"/>
</poll>
</flow>Daily reports are typically independent of incoming events and need to execute at a specific time.
Schedulers trigger flows based solely on the defined timing, making them suitable for recurring reporting tasks.
Poll Scopes are better for monitoring new data, so using them for daily fixed reports could introduce unnecessary complexity.
Thread pool configuration ensures that both Scheduler and Poll Scope can operate concurrently without blocking other flows.
Poll Scope should handle incremental data fetching to avoid duplicates, while Scheduler defines execution intervals.
Error handling and retries prevent data loss or repeated failures, maintaining flow reliability.
The cron expression schedules the flow for 9 AM, Monday through Friday.
The Try-Catch with Until Successful provides up to 2 retries with a 10-second interval for handling transient errors during execution.
// XML
<flow name="weekdaySchedulerFlow">
<scheduler doc:name="Scheduler" cronExpression="0 0 9 ? * MON-FRI"/>
<try doc:name="Try">
<logger message="Executing weekday task" level="INFO"/>
</try>
<catch-exception-strategy doc:name="Catch">
<until-successful maxRetries="2" millisBetweenRetries="10000">
<logger message="Retrying failed weekday task" level="WARN"/>
</until-successful>
</catch-exception-strategy>
</flow>The Poll Scope checks the FTP directory '/uploads' every 60 seconds for new files.
Logging allows tracking of newly detected files for further processing or auditing purposes.
// XML
<flow name="ftpPollFlow">
<poll doc:name="Poll" frequency="60000">
<ftp:list-remote-files config-ref="FTP_Config" directory="/uploads"/>
<logger message="New file detected: #[payload]" level="INFO"/>
</poll>
</flow>Backpressure occurs when the Poll Scope fetches more messages than downstream components can handle, causing memory pressure and potential slowdowns.
Strategies include using smaller batch sizes, introducing throttling, leveraging asynchronous processing, or storing messages temporarily in queues or object stores.
Proper monitoring and metrics allow identifying and adjusting polling frequency to balance throughput and system stability.
Tracking the last processed record ensures new polls only fetch unseen records.
Object Store allows persistence of state between polls, helping prevent duplicates.
Transactional boundaries ensure partial failures do not commit, avoiding duplicate processing.
The flow triggers every 3600000 milliseconds (1 hour).
The logger prints the current timestamp each time the Scheduler executes.
// XML
<flow name="hourlySchedulerFlow">
<scheduler doc:name="Scheduler" frequency="3600000"/>
<logger message="Flow executed at #[now()]" level="INFO"/>
</flow>When multiple Poll Scopes fetch data from different systems, coordination is critical to avoid race conditions or conflicting updates.
Techniques include controlling thread pool allocations, using asynchronous processing with queues, and implementing locks or transactional boundaries where required.
Dependencies between flows should be modeled explicitly to prevent conflicts and ensure consistent state across systems.
Retries ensure that transient failures do not prevent critical tasks from completing.
Object Store persistence allows tracking of processed items or state between executions.
Error handling strategies capture exceptions and prevent flow crashes, improving reliability.
The Poll Scope triggers every 15 seconds to fetch JSON data.
DataWeave filters only records where status is 'active', ensuring downstream processing handles relevant items.
// XML
<flow name="apiActivePollFlow">
<poll doc:name="Poll" frequency="15000">
<http:request config-ref="HTTP_Request_Config" method="GET" url="https://api.example.com/items"/>
<dw:transform-message doc:name="Filter Active">
<dw:set-payload><![CDATA[%dw 2.0
output application/json
---
payload filter $.status == 'active']]></dw:set-payload>
</dw:transform-message>
<logger message="Active items: #[payload]" level="INFO"/>
</poll>
</flow>A fixed-frequency Scheduler triggers flows at regular intervals, e.g., every 5 minutes, regardless of the time of day.
A cron-based Scheduler allows more complex scheduling, such as triggering at specific hours, weekdays, or months, using cron expressions.
Cron-based scheduling is ideal for tasks tied to business hours or specific calendar dates, while fixed-frequency is simpler for recurring short-interval tasks.
Object Store persistence allows Poll Scopes to track last processed items without reprocessing.
Lower polling frequency and smaller batches reduce load on source systems and Mule runtime.
Asynchronous processing allows concurrent handling of multiple poll results, improving throughput.
The Poll Scope monitors the '/incoming' directory on the SFTP server every 45 seconds.
Logging tracks new files, enabling further processing or integration workflows.
// XML
<flow name="sftpPollFlow">
<poll doc:name="Poll" frequency="45000">
<sftp:list config-ref="SFTP_Config" path="/incoming"/>
<logger message="Detected SFTP file: #[payload]" level="INFO"/>
</poll>
</flow>Cron expression schedules the flow for midnight on the last day of every month.
The Until Successful scope retries up to 3 times at 10-second intervals if execution fails, ensuring critical monthly tasks complete reliably.
// XML
<flow name="monthlySchedulerFlow">
<scheduler doc:name="Scheduler" cronExpression="0 0 0 L * ?"/>
<try doc:name="Try">
<logger message="Executing end-of-month task" level="INFO"/>
</try>
<catch-exception-strategy doc:name="Catch">
<until-successful maxRetries="3" millisBetweenRetries="10000">
<logger message="Retrying failed monthly task" level="WARN"/>
</until-successful>
</catch-exception-strategy>
</flow>