Python 簡單的平行處理任務

今天來記錄一下 Python 平行程式的用法

Python 有三種方式可以平行執行程式

1. multiprocess
2. threading
3. asyncio

以下分別是三種方式的範例程式碼

multiprocess

import multiprocessing
import os
import time


def compute_square(number):
    process_name = multiprocessing.current_process().name
    process_id = os.getpid()
    print(
        f"Process {process_name} (process id: {process_id}) computing square of {number}"
    )
    time.sleep(1)
    print(f"Result: {number}^2 = {number * number}")


if __name__ == "__main__":
    numbers = [2, 4, 6, 8]
    processes = []
    for num in numbers:
        p = multiprocessing.Process(target=compute_square, args=(num,))
        processes.append(p)
        p.start()
    for p in processes:
        p.join()
    print("All processes finished.")

  

threading

import os
import threading
import time


def compute_square(number):
    thread_name = threading.current_thread().name
    thread_id = threading.get_ident()
    process_id = os.getpid()
    print(
        f"Thread {thread_name} (thread id: {thread_id}, process id: {process_id}) computing square of {number}"
    )
    time.sleep(1)
    print(f"Result: {number}^2 = {number * number}")


numbers = [2, 4, 6, 8]
threads = []

for num in numbers:
    t = threading.Thread(target=compute_square, args=(num,))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

print("All threads finished.")

  

asyncio

import asyncio
import os
import threading


async def compute_square(number):
    task = asyncio.current_task()
    task_id = id(task)
    thread_id = threading.get_ident()
    process_id = os.getpid()
    print(
        f"Asyncio Task (id: {task_id}) running in thread id: {thread_id}, process id: {process_id}, computing square of {number}"
    )
    await asyncio.sleep(1)
    print(f"Result: {number}^2 = {number * number}")


async def main():
    numbers = [2, 4, 6, 8]
    tasks = [compute_square(num) for num in numbers]
    await asyncio.gather(*tasks)
    print("All asyncio tasks finished.")


if __name__ == "__main__":
    asyncio.run(main())

  

從這三個範例可以看到執行的地方不同：

1. multiprocessing 是在不同的 process 執行的。
2. threading 是在同一個 process 的不同 thread 執行的。
3. asyncio 是在同一個 process 的同一個 thread 執行的。

這三個範例都是簡單的執行一個單純的 task ，所以可以簡單的平行處理，如果 task 之間有順序性或是需要交換資料，都會讓平行處理的複雜度上升。