Bug: UserDataStreamEventsResponse can not perform model_validate_json correctly

[yagggi]

Description:

UserDataStreamEventsResponse can't serialize data properly, and this makes DerivativesTradingUsdsFuturesWebSocketStreams.user_data not working as expected.

Steps to reproduce:

import asyncio
import os
import logging

from binance_sdk_derivatives_trading_usds_futures.derivatives_trading_usds_futures import (
    DerivativesTradingUsdsFutures,
    DERIVATIVES_TRADING_USDS_FUTURES_WS_STREAMS_PROD_URL,
    ConfigurationWebSocketStreams,
)


# Configure logging
logging.basicConfig(level=logging.INFO)

# Create configuration for the WebSocket Streams
configuration_ws_streams = ConfigurationWebSocketStreams(
    stream_url=os.getenv(
        "STREAM_URL", DERIVATIVES_TRADING_USDS_FUTURES_WS_STREAMS_PROD_URL
    )
)

# Initialize DerivativesTradingUsdsFutures client
client = DerivativesTradingUsdsFutures(config_ws_streams=configuration_ws_streams)

async def run(listen_key):
    conn = None
    try:
        conn = await client.websocket_streams.create_connection()
        user_data_stream = await conn.user_data(listenKey=listen_key)
        user_data_stream.on("message", lambda x: print(f'{x}'))
        while True:
            await asyncio.sleep(0)
    except Exception as e:
        logging.error(e, exc_info=True)
    finally:
        if conn:
            await conn.close_connection(close_session=True)

Which gives outputs:

oneof_schema_1_validator=None oneof_schema_2_validator=None oneof_schema_3_validator=None oneof_schema_4_validator=None oneof_schema_5_validator=None oneof_schema_6_validator=None oneof_schema_7_validator=None oneof_schema_8_validator=None oneof_schema_9_validator=None actual_instance=None one_of_schemas={'StrategyUpdate', 'ConditionalOrderTriggerReject', 'MarginCall', 'Listenkeyexpired', 'TradeLite', 'GridUpdate', 'OrderTradeUpdate', 'AccountConfigUpdate', 'AccountUpdate'} discriminator_value_class_map={}

Possible root cause:

Stream subscriptions are handled by here, but seems UserDataStreamEventsResponse can not perform model_validate_json properly:

In [4]: json.dumps(d)
Out[4]: '{"e": "ORDER_TRADE_UPDATE", "E": 1568879465651, "T": 1568879465650, "o": {"s": "BTCUSDT", "c": "TEST", "S": "SELL", "o": "TRAILING_STOP_MARKET", "f": "GTC", "q": "0.001", "p": "0", "ap": "0", "sp": "7103.04", "x": "NEW", "X": "NEW", "i": 8886774, "l": "0", "z": "0", "L": "0", "N": "USDT", "n": "0", "T": 1568879465650, "t": 0, "b": "0", "a": "9.91", "m": false, "R": false, "wt": "CONTRACT_PRICE", "ot": "TRAILING_STOP_MARKET", "ps": "LONG", "cp": false, "AP": "7476.89", "cr": "5.0", "pP": false, "si": 0, "ss": 0, "rp": "0", "V": "EXPIRE_TAKER", "pm": "OPPONENT", "gtd": 0}}'

In [5]: OrderTradeUpdate.model_validate(d)
Out[5]: OrderTradeUpdate(E=1568879465651, T=1568879465650, o=OrderTradeUpdateO(s='BTCUSDT', c='TEST', S='SELL', o='TRAILING_STOP_MARKET', f='GTC', q='0.001', p='0', ap='0', sp='7103.04', x='NEW', X='NEW', i=8886774, l='0', z='0', L='0', N='USDT', n='0', T=1568879465650, t=0, b='0', a='9.91', m=False, R=False, wt='CONTRACT_PRICE', ot='TRAILING_STOP_MARKET', ps='LONG', cp=False, AP='7476.89', cr='5.0', pP=False, si=0, ss=0, rp='0', V='EXPIRE_TAKER', pm='OPPONENT', gtd=0, additional_properties={}), additional_properties={})

In [6]: UserDataStreamEventsResponse.model_validate_json(json.dumps(d))
Out[6]: UserDataStreamEventsResponse(oneof_schema_1_validator=None, oneof_schema_2_validator=None, oneof_schema_3_validator=None, oneof_schema_4_validator=None, oneof_schema_5_validator=None, oneof_schema_6_validator=None, oneof_schema_7_validator=None, oneof_schema_8_validator=None, oneof_schema_9_validator=None, actual_instance=None, one_of_schemas={'AccountConfigUpdate', 'MarginCall', 'Listenkeyexpired', 'AccountUpdate', 'StrategyUpdate', 'ConditionalOrderTriggerReject', 'TradeLite', 'OrderTradeUpdate', 'GridUpdate'}, discriminator_value_class_map={})

[iswarm]

The log output indicates an error when attempting to connect to Binance's WebSocket API (at wss://ws-api.binance.com:443/ws-api/v3) and perform a user data stream ping operation. The specific issue is error code -1102 from the server, which means the mandatory 'apiKey' parameter was not included in the WebSocket request (or it was empty, null, or malformed). This causes the connection to close immediately, and the process exits with code 0.
Despite having correct API key and secret values visible in debugging, and successfully generating the client and listenKey via REST API methods (new_user_data_stream() and put_user_data_stream()), the problem persists with user_data_stream_start() as well. This suggests that the WebSocket API client (used for these ping and start operations) was not initialized with the API key (and possibly the secret, though the secret is not strictly required for non-signed endpoints like user data stream ping). In the binance-connector-python library, the WebSocket API (distinct from the REST Spot client) requires the api_key to be passed explicitly for private requests, as it includes the 'apiKey' in the JSON params sent to the server.

Connected to pydev debugger (build 251.26094.141) INFO:root:Connecting to wss://ws-api.binance.com:443/ws-api/v3 with proxy None INFO:root:Establishing Websocket connection with id 1426776e-d3a3-423b-a605-eb9fec8cda40 to: wss://ws-api.binance.com:443/ws-api/v3 WARNING:root:Connection with user closed: Error received from server: {'code': -1102, 'msg': "Mandatory parameter 'apiKey' was not sent, was empty/null, or malformed."} INFO:root:user_data_stream_ping() rate limits: [] INFO:root:user_data_stream_ping() response: {'error': 'Error received from server: {'code': -1102, 'msg': "Mandatory parameter 'apiKey' was not sent, was empty/null, or malformed."}'} INFO:root:WebSocket 1426776e-d3a3-423b-a605-eb9fec8cda40 closed. Process finished with exit code 0

[rocky-d]

Pull request #423 fixed this bug, waiting to be reviewed and merged.

[alplabin]

Hey @yagggi,
Thanks for reporting this issue. A new release of binance-common (v4.4.0) has been made which fix this issue.
User data stream events are now matched to their schema, so ORDER_TRADE_UPDATE reaches your callback as a typed model instead of a raw dict. Let us know if you still see the issue.