Why does Python not use 100% of the processor?
I am developing a Python application where in some snippets, it demands a lot of CPU for calculations.
However, I notice that even at these "bottleneck" points, the CPU never reaches more than 50% usage.
Naturally the program gets slower, while it has CPU "to spare" that could make it faster.
For example, the process below takes (on my PC), 15 seconds :
from math import *
import time
ini = time.time()
for x in range(10**8):
a = cos(x)
print ("Tempo Total: ", time.time() - ini)
But during processing, only a few logical processors are used, and only 1 processor suffers more demand, yet it does not reach 100%:
How to make Python use 100% CPU in critical processes?
1 answers
In your case what is happening is as follows, you are running your routine on a single thread on a system (CPU) that has 4 logical cores.
You will need to rewrite your algorithm to be multi-threaded , or else check if you can run different instances on specific cores. I don't know any other way.
Come on..
I set up an example with tabular calculation to show the performance difference between each of the scenarios.
Scenario 1-thread-free execution
import time
ini = time.time()
def tabuada( threadName, numero):
count = 0
while count < 300000000:
count +=1
#print(" %s x %s = %s\n" % (numero, count, count*numero))
tabuada("Thread-1",2)
tabuada("Thread-1",3)
tabuada("Thread-1",4)
print ("Tempo Total: ", time.time() - ini)
In this execution I had result:
Total Time: 35.65071749687195
Scenario 2 - run with thread running parallel
import time
import _thread
ini = time.time()
def tabuada( threadName, numero):
count = 0
while count < 100000000:
count +=1
#print(" %s x %s = %s\n" % (numero, count, count*numero))
print ("Tempo Total: ", time.time() - ini)
def tabuada2( threadName, numero):
count = 0
while count < 100000000:
count +=1
#print(" %s x %s = %s\n" % (numero, count, count*numero))
print ("Tempo Total: ", time.time() - ini)
def tabuada3( threadName, numero):
count = 0
while count < 100000000:
count +=1
#print(" %s x %s = %s\n" % (numero, count, count*numero))
print ("Tempo Total: ", time.time() - ini)
try:
_thread.start_new_thread( tabuada, ("Thread12",2,) )
_thread.start_new_thread( tabuada2, ("Thread14",3,) )
_thread.start_new_thread( tabuada2, ("Thread15",4,) )
except:
print('Erro')
In this execution I had result:
Total Time: 10.11058759689331
Total Time: 11.129863500595093
Total Time: 11.548049688339233
Obs. Time total in this case was 11.54 s, since, this was the time that took the longest one thread. And the total time displayed on each row represents how long each thread took in its execution.
How can we notice the time drops a lot when we divide the task, but why? Because when we create different threads the OS interprets as different processes and allocates in different locations and cores within my CPU.
Being so as I said at the beginning, if you want use more your CPU and optimize processing time I advise you to use threads
Follows two nice links about the content: