fix mlserver infer with BYTES

[RafalSkolasinski]

Closes #1210

[RafalSkolasinski]

I have the feeling this should be something handled in the NumpyCodec but it may be too tritonclient specific...

[adriangonz]

Good point - do you know what's returned by NumpyCodec.decode_input at the moment?

[RafalSkolasinski]

Tritonclient expects

    elif np_dtype == np.object_ or np_dtype.type == np.bytes_:
        return "BYTES"
    return None

as otherwise it is None that was causing an issue.

As we do not set anything there explicitly (and I also seen that by adding extra logs earlier during debugging) we just have '<U1'

In [2]: import numpy as np

In [3]: x = np.array(["x"])

In [4]: x.dtype == np.bytes_
Out[4]: False

In [5]: x.dtype
Out[5]: dtype('<U1')

[adriangonz]

I think what's going is that the NumpyCodec encodes that list as a tensor of strings (i.e. using np.dtype(str)). However, Triton doesn't know how to handle that case.

MLServer/mlserver/codecs/numpy.py

Lines 41 to 43 in d283fc0

	if is_list_of(data, str):
	# Handle special case of strings being treated as Numpy arrays
	return np.dtype(str)

I can't remember why we had to add that np.dtype(str) special case, but I would be keen to keep it as is to avoid any potential side effects. Having said that, it's unclear whether changing the dtype here may cause any issues downstream (although I'd expect that strings in np arrays are quite an edge case).

[adriangonz]

Nice one @RafalSkolasinski! Added a small comment around tests - I think we can just reuse one of the existing fixtures, but everything else looks great. 👍

[adriangonz]

Good point - do you know what's returned by NumpyCodec.decode_input at the moment?

[adriangonz]

Thanks for tweaking the fixture @RafalSkolasinski ! This should now be ready to go once the linter stops complaining (seems to be a black formatting issue - you can fix it with make fmt).

[RafalSkolasinski]

@adriangonz done - should be fine now