Redirect stdout, stderr to websocket

[FabrizioRollo]

Hi,

I'm trying to redirect the stdout and stderr of each API call to a websocket or eventually a log file, here the code I'm using:

import meraki
from contextlib import redirect_stdout, redirect_stderr
import io
from flask import Flask
from flask_socketio import SocketIO, emit

async_mode = None

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, async_mode=async_mode)

@app.route('/getOrganizations')
def getOrganizations():
    captured_output = io.StringIO()
    with redirect_stdout(captured_output), redirect_stderr(captured_output):
        try:
            API_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            dashboard = meraki.DashboardAPI(API_KEY, output_log=False)

            organizations = dashboard.organizations.getOrganizations()
            socketio.emit('my_test',
                          {'data': captured_output.getvalue()})
            return {'organizations': organizations}

        except meraki.APIError as err:
            print('Error: ', err)
            return {'error': err}

If I restart the Flask server everything works fine the FIRST call and I get the desired output:

2022-05-05 17:02:32 meraki: INFO > Meraki dashboard API session initialized with these parameters: {'version': '1.15.0', 'api_key': '************************************9ea0', 'base_url': 'https://api.meraki.com/api/v1', 'single_request_timeout': 60, 'certificate_path': '', 'requests_proxy': '', 'wait_on_rate_limit': True, 'nginx_429_retry_wait_time': 60, 'action_batch_retry_wait_time': 60, 'retry_4xx_error': False, 'retry_4xx_error_wait_time': 60, 'maximum_retries': 2, 'simulate': False, 'be_geo_id': None, 'caller': None, 'use_iterator_for_get_pages': False} 2022-05-05 17:02:32 meraki: DEBUG > {'tags': ['organizations', 'configure'], 'operation': 'getOrganizations', 'method': 'GET', 'url': '/organizations', 'params': None} 2022-05-05 17:02:32 meraki: INFO > GET https://api.meraki.com/api/v1/organizations 2022-05-05 17:02:33 meraki: INFO > GET https://n392.meraki.com/api/v1/organizations 2022-05-05 17:02:34 meraki: INFO > organizations, getOrganizations - 200 OK

BUT in the subsequent calls nothing will be redirect to captured_output, it returns just nothing!

I've tried with different methods eg. sys.stdout, sys.stderr, with websocket or redirect to file, Flask, FastAPI...you name it!
I was able to get the stdout/stderr only the first time after a server restart.

Has someone an idea?

Regards

Fabrizio

[RaphaelL]

Hi , 

I'm really not familiar with your setup and Flask , but in your code there is only one call which is 

dashboard.organizations.getOrganizations

Where are the other calls ? Where they omited ?

[FabrizioRollo]

@RaphaelLI'm calling the endpoint via GUI with a button, subsequent calls to the same endpoints just return nothing.

here's the GUI meraki-explorer 

[PhilipDAth]

I have no idea, but have you thought about using one of the existing Python logging modules?  The Meraki Python module also has a lot of logging controls for things it does.  You could see which logging module it uses and copy.

[FabrizioRollo]

@PhilipDAthpython has a logging handler for socket SocketHandler but this is TCP socket handler, not a websocket one, adapting this handler to my needs was not the best option for me.

[Greenberet]

the Async/DashboardAPI objects are saving the logger in the _logger member.

You could just do a

dashboard._logger.addHandler( my_logger_handler ) 

then all messages from which are sent through the logging module would be sent to your handler

[FabrizioRollo]

@Greenberetthis could have been a solution if I had a couple of dashboard instances, since my script has several dashboard instances i preferred to a "global" solution as explained below. Anyway THX

[FabrizioRollo]

I was finally able to address the issue, the problem was how io.String() handles the buffer...but I ended up with a complete other solution.
I was looking to catch the API execution logs and display it in the web Application console, the best solution is to redirect the stdout/stderr to a redis pub/sub server like this:

from rlog import RedisHandler

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s %(name)12s: %(levelname)8s > %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
)

logger = logging.getLogger()
logger.addHandler(RedisHandler(channel='live_log',host=redis_hostname, port=6379))

The RedisHandler forwards the stdout/stderr to redis, a websocket server then send all the output to its connected clients (the webapplication console).
The result is a "live log" console that lets you follow the script execution without waiting for the API response.
Final result can be seen here
If you have already installed  the meraki-explorer don't forget to pull the new changes.

[PhilipDAth]

That is good lateral thinking!