Slack bolt installation to workspace fails if workspace name is in Japanese

[joseph-scu]

Hello,

I am trying to install my Slack Bolt app to a certain workspace that contains a Japanese workspace as its name (for testing purposes I used 撮影課). I am using OAuth for authentication. When I attempt to install the app I get a 502 Bad Gateway Error. There doesn't seem to be any errors in the logs, however I noticed if I tried to use the app in Slack I do see an error (see below).

My code for hanlding OAUth is as below (I believe its taken mostly from the docs)

load_dotenv()

try:
    with engine.connect() as conn:
        conn.execute("select count(*) from slack_bots")
except Exception as e:
    installation_store.metadata.create_all(engine)
    oauth_state_store.metadata.create_all(engine)

oauth_settings = OAuthSettings(
    client_id=os.environ.get("SLACK_CLIENT_ID"),
    client_secret=os.environ.get("SLACK_CLIENT_SECRET"),
    scopes=["channels:history", "channels:join", "channels:read",
            "chat:write", "commands", "groups:history", "im:history", "users:read"],
    installation_store=installation_store,
    state_store=oauth_state_store,
    install_page_rendering_enabled=False
)

app = App(
    signing_secret=os.environ.get("SLACK_SIGNING_SECRET"),
    oauth_settings=oauth_settings
)

Reproducible in:

As below

The slack_bolt version

alembic==1.9.2
apt-xapian-index==0.49
attrs==22.2.0
autopep8==2.0.1
awscli==2.9.19
awscrt==1.0.0.dev0
catfish==4.16.4
certifi==2022.12.7
chardet==5.1.0
charset-normalizer==3.0.1
colorama==0.4.6
cryptography==38.0.4
cupshelpers==1.0
dbus-python==1.3.2
deprecation==2.0.7
distro==1.8.0
docutils==0.19
dtrx==8.4.0
gattlib==0.20210616
greenlet==2.0.1
gyp==0.1
idna==3.3
iniconfig==1.1.1
jmespath==1.0.1
lightdm-gtk-greeter-settings==1.2.2
lxml==4.9.2
Mako==1.2.4
MarkupSafe==2.1.2
more-itertools==8.10.0
mugshot==0.4.3
mypy==0.991
mypy-extensions==0.4.3
notify2==0.3
ntpsec==1.2.2
onboard==1.4.1
packaging==23.0
pdfarranger==1.9.2
pexpect==4.8.0
pikepdf==6.0.0+dfsg
Pillow==9.4.0
pluggy==1.0.0+repack
prompt-toolkit==3.0.38
ptyprocess==0.7.0
py==1.11.0
pyasn1==0.4.8
PyBluez==0.23
pycairo==1.20.1
pycodestyle==2.10.0
pycups==2.0.1
PyGObject==3.42.2
pysmbc==1.0.23
pytest==7.2.1
python-apt==2.6.0
python-dateutil==2.8.2
python-debian==0.1.49
pyxdg==0.28
PyYAML==6.0
requests==2.28.2
roman==3.3
rpl==1.14
ruamel.yaml==0.17.21
ruamel.yaml.clib==0.2.7
six==1.16.0
SQLAlchemy==2.0.0
systemd-python==235
typing_extensions==4.4.0
ufw==0.36.2
urllib3==1.26.12
wcwidth==0.2.5
xdg==5

Python runtime version

Python 3.11.4

OS info

#1 SMP PREEMPT_DYNAMIC Debian 6.0.10-2 (2022-12-01)

Steps to reproduce:

(Share the commands to run, source code, and project settings (e.g., setup.py))

1. Have a workspace with Japanese characters.
2. Ensure you are using OAuth for authentication
3. Attempt to install the application (via /slack/install)
4. Notice that you will get a 502 Bad Gateway Error. The app will be visible in your list of apps however, but it won't be accessible

Expected result:

App installs to workspace

Actual result:

Application installs to the workspace, but not successfully. There is a 502 Bad Gateway Error, but the app is visible in workspaces list of apps. If you try to navigate to it you may see the following error in the logs:

return codecs.charmap_encode(input,errors,encoding_table)
| UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2: character maps to <undefined>

[seratch]

Hi @joseph-scu, thanks for asking the question! However, with the limited information, we may not be able to help you out. I'd suggest enabling debug-level logging to see more details of the error first: https://slack.dev/bolt-python/concepts#logging

Also, you can share more details such as:

• slack-bolt, slack-sdk versions
• python runtime version / OS version
• the infra you're running the app (e.g., AWS, Linux, whatever could be helpful for identifying the cause)
• the code snippet you're using now

You can use our issue template to fill out these information (for future issue submissions, please use the template): https://github.com/slackapi/bolt-python/blob/main/.github/ISSUE_TEMPLATE/01_question.md

[seratch]

For your information, I have a Slack workspace with Japanese name for my personal use and I've installed a few bolt-python apps into it. However, I've never experienced the issue. Thus, just having the Japanese name may not be the cause of your issue.

[joseph-scu]

@seratch Thanks for that, I've updated my question. The reason I suspected it was due to the Japanese name was because the app installs fine if it is in English.

[seratch]

Would it be possible for you to identify where the UnicodeEncodeError is raised?

[joseph-scu]

This was the best I could do, I apologize for the long stack trace:

Exception happened during processing of request from ('172.20.0.1', 36920)
ku-slack-1  | Traceback (most recent call last):
ku-slack-1  |   File "/usr/local/lib/python3.8/socketserver.py", line 316, in _handle_request_noblock
ku-slack-1  |     self.process_request(request, client_address)
ku-slack-1  |   File "/usr/local/lib/python3.8/socketserver.py", line 347, in process_request
ku-slack-1  |     self.finish_request(request, client_address)
ku-slack-1 |   File "/usr/local/lib/python3.8/socketserver.py", line 360, in finish_request
ku-slack-1  |     self.RequestHandlerClass(request, client_address, self)
ku-slack-1  |   File "/usr/local/lib/python3.8/http/server.py", line 664, in __init__
ku-slack-1  |     super().__init__(*args, **kwargs)
ku-slack-1  |   File "/usr/local/lib/python3.8/socketserver.py", line 747, in __init__
ku-slack-1  |     self.handle()
ku-slack-1  |   File "/usr/local/lib/python3.8/http/server.py", line 435, in handle
ku-slack-1  |     self.handle_one_request()
ku-slack-1  |   File "/usr/local/lib/python3.8/http/server.py", line 423, in handle_one_request
ku-slack-1  |     method()
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/slack_bolt/app/app.py", line 1349, in do_GET
ku-slack-1  |     bolt_resp = _bolt_oauth_flow.handle_callback(bolt_req)
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/slack_bolt/oauth/oauth_flow.py", line 278, in handle_callback
ku-slack-1  |     self.store_installation(request, installation)
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/slack_bolt/oauth/oauth_flow.py", line 361, in store_installation
ku-slack-1  |     self.settings.installation_store.save(installation)
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/slack_sdk/oauth/installation_store/sqlalchemy/__init__.py", line 158, in save
ku-slack-1  |     conn.execute(self.installations.insert(), i)
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
ku-slack-1  |     return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
ku-slack-1  |     return connection._execute_clauseelement(
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
ku-slack-1  |     ret = self._execute_context(
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
ku-slack-1  |     self._handle_dbapi_exception(
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2128, in _handle_dbapi_exception
ku-slack-1  |     util.raise_(exc_info[1], with_traceback=exc_info[2])
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
ku-slack-1  |     raise exception
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
ku-slack-1  |     self.dialect.do_execute(
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
ku-slack-1  |     cursor.execute(statement, parameters)
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 199, in execute
ku-slack-1  |     args = tuple(map(db.literal, args))
ku-slack-1  |   File "/usr/local/lib/python3.8/site-packages/MySQLdb/connections.py", line 275, in literal
ku-slack-1  |     s = self.string_literal(o.encode(self.encoding))
ku-slack-1  |   File "/usr/local/lib/python3.8/encodings/cp1252.py", line 12, in encode
ku-slack-1  |     return codecs.charmap_encode(input,errors,encoding_table)
ku-slack-1  | UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2: character maps to <undefined>

[seratch]

It seems that the error occurs during SQLAlchemy's database operations with MySQL. You can try the answer here: https://stackoverflow.com/questions/63220132/sqlalchemy-insert-to-mysql-db-unicodeencodeerror-for-cyrlic-data

[joseph-scu]

@seratch Thank you very much for providing the link!

I changed my engine_url to the following:

engine_url = URL.create(
    os.environ.get("DB_DRIVER", ""),
    username=os.environ.get("DB_USERNAME"),
    password=os.environ.get("DB_PASSWORD"),
    host=os.environ.get("DB_HOST"),
    database=os.environ.get("DB_DATABASE")
)

engine_url = engine_url.update_query_pairs([("charset", "utf8mb4")])

However, this alone was not enough. I also had to modify the character set of team_name in both the slack_bots and slack_installation columns to utf8mb4, as well as collation to utf8mb4_unicode_ci. With these changes it seems to install as it should.

[seratch]

@joseph-scu Thanks for sharing this. It seems that the built-in SQLAlchemy-based installation store (https://github.com/slackapi/python-slack-sdk/blob/v3.23.0/slack_sdk/oauth/installation_store/sqlalchemy/__init__.py) can be improved to better support MySQL / MariaDB in future versions. I will check it out when I have the bandwidth for it.

Since you've identified the cause and resolved the issue, let me close this issue now. Thanks again for writing in!