Odoo的异步队列神器-queue_job
-
你可以选择这个job，在action中选'requeue', 重新加入队列。
@digitalsatori 估计跟如下报错有关,但是我根据报错位置打印出来
dispatch是None.源码中这个值应该是这么来的
odoo.multi_process or odoo.evented.这个是跟设置还是什么有关还是其他,指点下?
2020-03-11 09:41:10,840 27721 DEBUG aa_v11 odoo.modules.registry: Multiprocess signaling check: [Registry - 13 -> 13] [Cache - 33 -> 33] 2020-03-11 09:41:10,842 27721 ERROR aa_v11 odoo.http: Exception during JSON request handling. Traceback (most recent call last): File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 653, in _handle_exception return super(JsonRequest, self)._handle_exception(exception) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 312, in _handle_exception raise pycompat.reraise(type(exception), exception, sys.exc_info()[2]) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/tools/pycompat.py", line 87, in reraise raise value File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 695, in dispatch result = self._call_function(**self.params) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 344, in _call_function return checked_call(self.db, *args, **kwargs) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/service/model.py", line 97, in wrapper return f(dbname, *args, **kwargs) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 337, in checked_call result = self.endpoint(*a, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 939, in __call__ return self.method(*args, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 517, in response_wrap response = f(*args, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/addons/bus/controllers/main.py", line 40, in poll raise Exception("bus.Bus unavailable") Exception: bus.Bus unavailable -
你贴出来的exception跟queue.job没关系，多进程模式，不用反向代理转发长连接请求都有这个错误，没有影响的。
你要检查Odoo服务器启动时，正常的话会显示job.queue worker启动的log信息
-
@digitalsatori 从log看,有WorkCorn的日志,但是报错一直是这个错误在循环,走不下去.
2020-03-12 02:18:43,989 6106 INFO aa_v11 werkzeug: 127.0.0.1 - - [12/Mar/2020 02:18:43] "POST /longpolling/poll HTTP/1.1" 200 - 2020-03-12 02:18:50,691 6111 DEBUG ? odoo.service.server: WorkerCron (6111) polling for jobs 2020-03-12 02:18:50,708 6111 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=aa_v11' 2020-03-12 02:18:50,708 6111 DEBUG ? odoo.service.server: WorkerCron (6111) aa_v11 time:0.005s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:50,709 6111 DEBUG ? odoo.service.server: WorkerCron (6111) polling for jobs 2020-03-12 02:18:50,720 6111 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=mm_v11' 2020-03-12 02:18:50,721 6111 DEBUG ? odoo.service.server: WorkerCron (6111) mm_v11 time:0.007s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:50,721 6111 DEBUG ? odoo.service.server: WorkerCron (6111) polling for jobs 2020-03-12 02:18:50,733 6111 WARNING ? odoo.addons.base.ir.ir_cron: Skipping database test10_v10 as its base version is not 11.0.1.3. 2020-03-12 02:18:50,734 6111 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=test10_v10' 2020-03-12 02:18:50,734 6111 DEBUG ? odoo.service.server: WorkerCron (6111) test10_v10 time:0.007s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:50,735 6111 DEBUG ? odoo.service.server: WorkerCron (6111) polling for jobs 2020-03-12 02:18:50,748 6111 WARNING ? odoo.addons.base.ir.ir_cron: Skipping database 2e_v8 as its base version is not 11.0.1.3. 2020-03-12 02:18:50,748 6111 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=2e_v8' 2020-03-12 02:18:50,749 6111 DEBUG ? odoo.service.server: WorkerCron (6111) 2e_v8 time:0.007s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:56,759 6113 DEBUG ? odoo.service.server: WorkerCron (6113) polling for jobs 2020-03-12 02:18:56,774 6113 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=aa_v11' 2020-03-12 02:18:56,775 6113 DEBUG ? odoo.service.server: WorkerCron (6113) aa_v11 time:0.007s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:56,775 6113 DEBUG ? odoo.service.server: WorkerCron (6113) polling for jobs 2020-03-12 02:18:56,788 6113 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=mm_v11' 2020-03-12 02:18:56,788 6113 DEBUG ? odoo.service.server: WorkerCron (6113) mm_v11 time:0.008s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:56,789 6113 DEBUG ? odoo.service.server: WorkerCron (6113) polling for jobs 2020-03-12 02:18:56,799 6113 WARNING ? odoo.addons.base.ir.ir_cron: Skipping database test10_v10 as its base version is not 11.0.1.3. 2020-03-12 02:18:56,799 6113 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=test10_v10' 2020-03-12 02:18:56,799 6113 DEBUG ? odoo.service.server: WorkerCron (6113) test10_v10 time:0.005s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:56,800 6113 DEBUG ? odoo.service.server: WorkerCron (6113) polling for jobs 2020-03-12 02:18:56,810 6113 WARNING ? odoo.addons.base.ir.ir_cron: Skipping database 2e_v8 as its base version is not 11.0.1.3. 2020-03-12 02:18:56,810 6113 INFO ? odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 1 connections to 'sslmode=prefer user=hui dbname=2e_v8' 2020-03-12 02:18:56,811 6113 DEBUG ? odoo.service.server: WorkerCron (6113) 2e_v8 time:0.006s mem: 330508.0k -> 330508.0k (diff: 0.0k) 2020-03-12 02:18:58,000 6107 DEBUG aa_v11 odoo.modules.registry: Multiprocess signaling check: [Registry - 14 -> 14] [Cache - 44 -> 44] 2020-03-12 02:18:58,003 6107 ERROR aa_v11 odoo.http: Exception during JSON request handling. Traceback (most recent call last): File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 653, in _handle_exception return super(JsonRequest, self)._handle_exception(exception) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 312, in _handle_exception raise pycompat.reraise(type(exception), exception, sys.exc_info()[2]) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/tools/pycompat.py", line 87, in reraise raise value File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 695, in dispatch result = self._call_function(**self.params) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 344, in _call_function return checked_call(self.db, *args, **kwargs) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/service/model.py", line 97, in wrapper return f(dbname, *args, **kwargs) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 337, in checked_call result = self.endpoint(*a, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 939, in __call__ return self.method(*args, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 517, in response_wrap response = f(*args, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/addons/bus/controllers/main.py", line 37, in poll raise Exception("bus.Bus unavailable") Exception: bus.Bus unavailable 2020-03-12 02:18:58,006 6107 INFO aa_v11 werkzeug: 127.0.0.1 - - [12/Mar/2020 02:18:58] "POST /longpolling/poll HTTP/1.1" 200 - 2020-03-12 02:18:59,679 6106 INFO ? werkzeug: 127.0.0.1 - - [12/Mar/2020 02:18:59] "GET /web/static/src/xml/dialog.xml?debug=1583979539675 HTTP/1.1" 200 - 2020-03-12 02:19:21,842 6106 DEBUG aa_v11 odoo.modules.registry: Multiprocess signaling check: [Registry - 14 -> 14] [Cache - 44 -> 44] 2020-03-12 02:19:21,844 6106 ERROR aa_v11 odoo.http: Exception during JSON request handling. Traceback (most recent call last): File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 653, in _handle_exception return super(JsonRequest, self)._handle_exception(exception) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 312, in _handle_exception raise pycompat.reraise(type(exception), exception, sys.exc_info()[2]) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/tools/pycompat.py", line 87, in reraise raise value File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 695, in dispatch result = self._call_function(**self.params) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 344, in _call_function return checked_call(self.db, *args, **kwargs) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/service/model.py", line 97, in wrapper return f(dbname, *args, **kwargs) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 337, in checked_call result = self.endpoint(*a, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 939, in __call__ return self.method(*args, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/odoo/http.py", line 517, in response_wrap response = f(*args, **kw) File "/home/hui/odoo-dev/odoo11/odoo11/addons/bus/controllers/main.py", line 37, in poll raise Exception("bus.Bus unavailable") Exception: bus.Bus unavailable -
你是不是没有按照queue.job模块的安装说明配置odoo的配置文件啊：
[options] (...) workers = 2 server_wide_modules = web,queue_job (...) [queue_job] channels = root:2 -
另外，你碰到的那个错误，我有写过一个帖子的：https://shine-it.net/topic/5701/odoo-错误-exception-bus-bus-unavailable-是什么鬼
-
你是不是没有按照queue.job模块的安装说明配置odoo的配置文件啊：
[options] (...) workers = 2 server_wide_modules = web,queue_job (...) [queue_job] channels = root:2@digitalsatori
非常感谢,现在通过官方的系统导入使用异步队列可以啦.我还有几个问题想再确认下:
1.worker设置多少合适
2.测试的时候发现,我用6500条数据(只有两个导入字段)测试,worker=2,系统导入的时候选择了后台导入,还使用了30min左右.速度没有提高多少
3.在odoo启动后创建的.(官方文档有,如果创建一个新数据库或在已有数据库上安装了job在不做任何操作的情况下,不会执行job,即使重写requeue也没效果,必须关闭重启odoo服务才会很快执行jobqueue_job,必须重启才会生效.)
3.如果自定义导入方法的话,就需要自己分割文件,生成不同的job了吧
4.多个channel和runner是怎么设置的,哪里设置的,什么作用 -
你是不是没有按照queue.job模块的安装说明配置odoo的配置文件啊：
[options] (...) workers = 2 server_wide_modules = web,queue_job (...) [queue_job] channels = root:2@digitalsatori
实验系统自带导入的时候启用异步导入,成功导入.但是速度没有快多少,什么原因?配置文件修改了如下的:
workers = 4 server_wide_modules = web,queue_job channels = root:1日志可以看到
queue.job启动了,2020-03-19 02:16:26,417 6384 INFO ? odoo.addons.queue_job.jobrunner.runner: starting 2020-03-19 02:16:26,418 6384 INFO ? odoo.addons.queue_job.jobrunner.runner: initializing database connections 2020-03-19 02:16:26,513 6384 DEBUG ? odoo.addons.queue_job.jobrunner.runner: queue_job is not installed for db ccp_v11 2020-03-19 02:16:26,542 6384 DEBUG ? odoo.addons.queue_job.jobrunner.runner: queue_job is not installed for db ll_v12 2020-03-19 02:16:26,653 6384 INFO ? odoo.addons.queue_job.jobrunner.runner: queue job runner ready for db queue2_v11 2020-03-19 02:16:26,728 6384 INFO ? odoo.addons.queue_job.jobrunner.runner: queue job runner ready for db queue_v11 2020-03-19 02:16:26,790 6384 INFO ? odoo.addons.queue_job.jobrunner.runner: queue job runner ready for db test_v11 2020-03-19 02:16:26,790 6384 INFO ? odoo.addons.queue_job.jobrunner.runner: database connections ready 2020-03-19 02:16:26,790 6384 DEBUG ? odoo.addons.queue_job.jobrunner.runner: select() timeout: 60.00 sec -
异步本身并不提高速度。要提高速度要考虑并发，并发是要将导入数据分割然后同时导入。但是并发的时候有时候要考虑并发锁。这个你要自己琢磨。
-
异步本身并不提高速度。要提高速度要考虑并发，并发是要将导入数据分割然后同时导入。但是并发的时候有时候要考虑并发锁。这个你要自己琢磨。
@digitalsatori
嗯嗯,好的,谢谢,非常感谢. -
@digitalsatori
非常感谢,现在通过官方的系统导入使用异步队列可以啦.我还有几个问题想再确认下:
1.worker设置多少合适
2.测试的时候发现,我用6500条数据(只有两个导入字段)测试,worker=2,系统导入的时候选择了后台导入,还使用了30min左右.速度没有提高多少
3.在odoo启动后创建的.(官方文档有,如果创建一个新数据库或在已有数据库上安装了job在不做任何操作的情况下,不会执行job,即使重写requeue也没效果,必须关闭重启odoo服务才会很快执行jobqueue_job,必须重启才会生效.)
3.如果自定义导入方法的话,就需要自己分割文件,生成不同的job了吧
4.多个channel和runner是怎么设置的,哪里设置的,什么作用@hui 请问，新建库重启问题如何处理的？有解决方案吗？
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login