Python 多線程 VS 多進程（三）

線程替代方案

subprocess

完全跳過線程，使用進程

是派生進程的主要替代方案

python2.4后引入

multiprocessiong

使用threading接口派生，使用子進程

允許為多核或者多cpu派生進程，接口跟threading非常相似

python2.6后引入

concurrent.futures

新的異步執行模塊

任務級別的操作

python3.2后引入

多進程

進程間通訊(InterprocessCommunication, IPC)

進程之間無任何共享狀態

進程的創建

直接生成Process實例對象，案例19

import multiprocessing from time import sleep, ctime def clock(interval): while True: print("The time is %s" % ctime()) sleep(interval) if __name__ == '__main__': p = multiprocessing.Process(target=clock, args=(5, )) p.start() while True: print("sleeping......") sleep(1)

sleeping...... The time is Tue Aug 13 19:47:41 2019 sleeping...... sleeping...... sleeping...... sleeping...... sleeping...... The time is Tue Aug 13 19:47:46 2019 sleeping...... sleeping...... sleeping...... sleeping...... ...

派生子類，案例20

import multiprocessing from time import sleep, ctime class ClockProcess(multiprocessing.Process): ''' 兩個函數比較重要 1. init構造函數 2. run ''' def __init__(self, interval): super(ClockProcess, self).__init__() self.interval = interval def run(self): while True: print("The time is %s" % ctime()) sleep(self.interval) if __name__ == '__main__': p = ClockProcess(3) p.start()

The time is Tue Aug 13 19:48:49 2019 The time is Tue Aug 13 19:48:52 2019 The time is Tue Aug 13 19:48:55 2019 The time is Tue Aug 13 19:48:58 2019 The time is Tue Aug 13 19:49:01 2019 The time is Tue Aug 13 19:49:04 2019 ...

在os中查看pid，ppid以及他們的關系

案例21

from multiprocessing import Process import os def info(title): print(title) print("module name:", __name__) # 得到父親進程的id print("parent process:", os.getppid()) # 得到本身進程的id print("process id:", os.getpid()) def f(name): info('function f') print('hello', name) if __name__ == '__main__': info('main line') p = Process(target=f, args=('bob', )) p.start() p.join()

main line module name: __main__ parent process: 11900 process id: 18832 function f module name: __mp_main__ parent process: 18832 process id: 20868 hello bob

生產者消費者模型

JoinableQueue

案例22

import multiprocessing from time import ctime def consumer(input_q): print("Into consumer:", ctime) while True: # 處理項 item = input_q.get() print('pull', item, 'out of q') input_q.task_done() # 發出信號通知任務完成 print("Out of consumer:", ctime()) ## 此句未執行，因為q.join()收集到四個task_done()信號后，主進程啟動，未等到print此句完成，程序就結束了 def producer(sequence, output_q): print("Into procuder:", ctime()) for item in sequence: output_q.put(item) print('put', item, 'into q') print('Out of procuder', ctime()) # 建立進程 if __name__ == '__main__': q = multiprocessing.JoinableQueue() # 進行消費者進程 cons_p = multiprocessing.Process(target=consumer, args=(q, )) cons_p.daemon = True cons_p.start() # 生產多個項，sequence代表要發送給消費者的項序列 # 在實踐中，這可能是生成器的輸出或通過一些其他地方生產出來 sequence = [1, 2, 3, 4] producer(sequence, q) # 等待所有項被處理 q.join()

Into procuder: Tue Aug 13 19:50:38 2019 put 1 into q put 2 into q put 3 into q put 4 into q Out of procuder Tue Aug 13 19:50:38 2019 Into consumer: pull 1 out of q pull 2 out of q pull 3 out of q pull 4 out of q

隊列中哨兵的使用，案例23

import multiprocessing from time import ctime # 設置哨兵問題 def consumer(input_q): print("Into consumer:", ctime()) while True: item = input_q.get() if item is None: break print("pull", item, "out of q") print ("Out of consumer:", ctime()) ## 此句執行完成，再轉入主進程 def producer(sequence, output_q): print ("Into procuder:", ctime()) for item in sequence: output_q.put(item) print ("put", item, "into q") print ("Out of procuder:", ctime()) if __name__ == '__main__': q = multiprocessing.Queue() cons_p = multiprocessing.Process(target = consumer, args = (q,)) cons_p.start() sequence = [1,2,3,4] producer(sequence, q) q.put(None) cons_p.join()

Into procuder: Tue Aug 13 19:51:23 2019 put 1 into q put 2 into q put 3 into q put 4 into q Out of procuder: Tue Aug 13 19:51:23 2019 Into consumer: Tue Aug 13 19:51:24 2019 pull 1 out of q pull 2 out of q pull 3 out of q pull 4 out of q Out of consumer: Tue Aug 13 19:51:24 2019

哨兵的改進，案例24

import multiprocessing from time import ctime def consumer(input_q): print ("Into consumer:", ctime()) while True: item = input_q.get() if item is None: break print("pull", item, "out of q") print ("Out of consumer:", ctime()) def producer(sequence, output_q): for item in sequence: print ("Into procuder:", ctime()) output_q.put(item) print ("Out of procuder:", ctime()) if __name__ == '__main__': q = multiprocessing.Queue() cons_p1 = multiprocessing.Process (target = consumer, args = (q,)) cons_p1.start() cons_p2 = multiprocessing.Process (target = consumer, args = (q,)) cons_p2.start() sequence = [1,2,3,4] producer(sequence, q) q.put(None) q.put(None) cons_p1.join() cons_p2.join()

Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into procuder: Tue Aug 13 19:52:08 2019 Out of procuder: Tue Aug 13 19:52:08 2019 Into consumer: Tue Aug 13 19:52:08 2019 pull 1 out of q pull 2 out of q pull 3 out of q pull 4 out of q Out of consumer: Tue Aug 13 19:52:08 2019 Into consumer: Tue Aug 13 19:52:08 2019 Out of consumer: Tue Aug 13 19:52:08 2019

Python 任務調度 多線程

版權聲明：本文內容由網絡用戶投稿，版權歸原作者所有，本站不擁有其著作權，亦不承擔相應法律責任。如果您發現本站中有涉嫌抄襲或描述失實的內容，請聯系我們jiasou666@gmail.com 處理，核實后本網站將在24小時內刪除侵權內容。