环境
Flask==2.2.2
Flask-SQLAlchemy==3.0.2
部分配置信息
class Config_base(object):
# 开启debug 生成模式请改成False
DEBUG = False
# log lever : DEBUG INFO WARNING ERROR FATAL
LOGLEVEL = 'DEBUG'
# 设置session密钥
SECRET_KEY = ''
# mysql
SQLALCHEMY_DATABASE_URI = "mysql://rpa_mysql:"+urllib.parse.quote_plus(
"123456")+"@192.168.1.1:3306/test_db"
# binds
SQLALCHEMY_BINDS = {
"doris-btg_marketing": "mysql://test:"+urllib.parse.quote_plus("123456")+"@192.168.1.1:9030/btg_marketing",
}
SQLALCHEMY_ECHO = False
SQLALCHEMY_RECORD_QUERIES = False
SQLALCHEMY_TRACK_MODIFICATIONS = False
FLASKY_DB_QUERY_TIMEOUT = 1
SQLALCHEMY_RECORD_QUERIES = True
SQLALCHEMY_COMMIT_ON_TEARDOWN = True
SQLALCHEMY_POOL_SIZE = 180
SQLALCHEMY_MAX_OVERFLOW = 20
SQLALCHEMY_POOL_TIMEOUT = 30
SQLALCHEMY_POOL_RECYCLE = 3600
SQLALCHEMY_ENGINE_OPTIONS = {
'pool_size': 80, # 连接池大小
'max_overflow': 20, # 连接池允许的最大溢出连接数
'pool_timeout': 30, # 连接池获取连接超时时间(秒)
'pool_recycle': 3600, # 连接池连接的生命周期(秒),在这之后连接会被回收
'pool_pre_ping': True # 启用连接池的预检功能,可以检测连接是否可用
}
# Flask-APScheduler
# 一定要开启API功能,这样才可以用api的方式去查看和修改定时任务
SCHEDULER_API_ENABLED = True
# 设置时区,时区不一致会导致定时任务的时间错误
SCHEDULER_TIMEZONE = 'Asia/Shanghai'
# api前缀(默认是/scheduler)
SCHEDULER_API_PREFIX = '/scheduler'
# 配置允许执行定时任务的主机名
SCHEDULER_ALLOWED_HOSTS = ['*']
故障现象
明明设置了180的连接池,多线程并发10个post请求函数(同时并发写入数据库)时,报错如下:
QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/14/3o7r)
初步诊断,SQLALCHEMY_POOL_SIZE 还是SQLALCHEMY_ENGINE_OPTIONS 设置未生效
然后google搜索相关内容:
SQLALCHEMY_ENGINE_OPTIONS only affects the default bind SQLALCHEMY_DATABASE_URI
Values in SQLALCHEMY_BINDS can be a dict with engine options for that bind, including the "url" key instead of only the connection string
https://flask-sqlalchemy.palletsprojects.com/en/stable/binds/#binds
github相关内容https://github.com/pallets-eco/flask-sqlalchemy/issues/799
github相关内容https://github.com/pallets-eco/flask-sqlalchemy/issues/783
解决办法
class Config_base修改SQLALCHEMY_BINDS(SQLALCHEMY_ENGINE_OPTIONS只对默认的SQLALCHEMY_DATABASE_URI生效,binds数据库需要SQLALCHEMY_BIND中单独指定参数)
# binds
SQLALCHEMY_BINDS = {
"doris-btg_marketing": {'url':"mysql://test:"+urllib.parse.quote_plus("123456")+"@192.168.1.1:9030/btg_marketing",'pool_size': 120,},
}