Changelog

v1.6.0

Full Changelog

Implemented enhancements:

  • Create a prepackaged model server for PyTorch Models #831

Fixed bugs:

  • IsADirectoryError: [Errno 21] Is a directory: ‘/mnt/models’ #2876

  • error: a container name must be specified for pod #2875

  • MLFlow server– ModuleNotFoundError: No module named ‘prediction’ #2874

  • V1 CRD has missing grpcPort and httpPort #2866

  • Broken Link to Documentation Example I’d like to find if it exists #2836

  • Executor does not send feedback to Routers. #2827

  • ArgoCD OutOfSync if SeldonDeployment includes mountpoint #2811

  • Helm failing to fetch https://kubernetes-charts.storage.googleapis.com/ resulting in failing tests #2808

  • send_feedback response is incorrectly managed in seldon_methods.py #2801

  • Upgrading to 1.5.0 causes unexpected error when calling predict endpoint of Python custom model #2786

  • SHAP Breaks Alibi Detect on Python 3.6 due to unpinned Numpy dependency #2767

  • Error when using the R language wrapper #2744

  • Transformers model unable to run with Cuda #2680

  • Allow seldon manager to run as non-root #2631

  • Operator sets HTTPS on the Engine’s liveness and ready checks #2586

  • high memory and cpu usage in deployment of xgboost rest #1986

Security fixes:

  • Resolve CVE for PyYAML - CVE-2020-14343 #2252

Closed issues:

  • CVE checks update for redhat image scans #2869

  • Does Seldon Batch Processing Work with Azure Blob Storage? #2858

  • Update engine docs as deprecated #2840

  • Support V2 Protocol in outlier and drift detectors #2831

  • add example of batch processor with rclone #2819

  • Add example of custom init container with rclone #2818

  • remove mutating webhook #2817

  • Handle KFServing V2 Protocol in request logger #2791

  • Create 1.5.1 release with cherrypick #2756

  • Use f-strings in MAB study case examples #2729

  • helm chart imagePullSecrets support to bypass ratelimiting #2694

  • Seldon-core-operator Update for handling namespace #2676

  • docs: No Release Highlights since 1.1.0 #2634

  • Depricate engine (old Java service orchestrator) #2588

  • Add support for Datadog Tracing in the Executor and the Python Wrapper #2436

  • Multi_Archtecture Support #2333

  • Make deployment names configurable #2301

  • java-wrapper-0.2.0 jar is not checked for validity #2180

  • Stateful Model Serving by Saving state to Redis #2138

  • Add documentation on how to extend base prepackaged servers with new images (xgboost, sklearn, etc) #2060

  • Add documentation that dives into the iniContainer #2055

  • Multiplexing or parallel serving of gRPC / REST in Python Wrapper #1968

  • Allow globally configurable docker registry secret for seldon deployments #1923

  • Remove probesonly flag #1856

  • Use custom errors #1841

  • Allow mixed rest/grpc graphs in new golang based executor #1820

Merged pull requests:

v1.5.1 (2020-12-17)

Full Changelog

Fixed bugs:

  • SC Operator continues to reconcile objects that are being (foreground) delete #2781

  • Custom metrics not available in Prometheus #2766

  • seldon-batch-processor on seldon-core-s2i-python37 image is not generating any output #2745

  • transport: is not respected on seldondeployment #2540

  • helm install results in wrong configmap #2528

Closed issues:

  • Hard requirement in Tensorflow (API) on GRPCIO 1.32.x breaks Seldon Core #2787

  • istio request timeouts #2727

  • Document how to run python wrapper locally for development #2722

  • Swagger API needs to be upgraded following best practices #2669

  • Authentication support for ELK Logging #2300

  • Support for xgboost4j-spark 0.9 #1395

Merged pull requests:

v1.5.0 (2020-12-03)

Full Changelog

Fixed bugs:

  • Can not create seldondSeldonDeployment crd in k8s v1.19.2 #2726

  • Custom metrics endpoint produces exceptions during load-testing #2704

  • Add a note in istio doc about pod security context for nonroot user #2686

  • KEDA notebook testing is broken #2683

  • [BUG] NodeSelector not working in SeldonDeployment #2682

  • minio setup notebook needs updating #2670

  • Alibi detect does not expose metrics when value is 0 #2668

  • Explainer wrapper should not add model to path for Tensorflow protocol #2664

  • Python processes when running Seldon with Gunicorn #2617

  • Update KEDA example to use v2.0 GA version #2614

  • Tutorial issues - CIFAR10 Drift Detection #2605

  • Conflict between gunicorn, gevent, and TensorFlow #2603

  • Fix notebook failing integartion tests for sklearn and xgboost V2 #2589

  • Repeatedly logging [DEBUG] Closing connection. #2568

  • Not able to pass string as input to the predict function. #2553

  • Notebook test test\_custom\_metrics failing in master #2541

  • grafana-prom-import-dashboards pod always fail in seldon-core-analytics chat #2518

  • requestPath meta missing in new executor #2505

  • ERROR: no endpoints available for service “seldon-webhook-service” while creating deployment using CUSTOM_INFERENCE_SERVER #2459

  • routing metadata missing in new executor #1823

Closed issues:

  • Seldon deployment support for google kubernetes ingress #2714

  • Using seldon-core-microservice how does one turn on SSL? #2706

  • Allow user to disable gunicorn access logs #2705

  • use service account for argo batch example #2673

  • Update metrics exposed by alibi detect server to include all newer components (threshold, etc) #2667

  • Add namespace to metrics component in seldon core python module #2666

  • Explore send_feedback path for tensorflow protocol #2665

  • More restricted deployment rbac for seldon-core #2662

  • GCP Workload Identity Support for GCS - Prepackaged Model Server #2654

  • Dry run deployment config #2648

  • seldon-batch-processor to support https and sslverify #2638

  • document how to use custom init containers #2610

  • Grafana on Ambassador (Public DNS)? #2591

  • Pip cache removal in wrappers/s2i/python/s2i/bin/assemble #2581

  • new knative filtering #2551

  • upgrade alibi-detect-server to use new alibi-detect when released #2405

  • Support gRPC and HTTP protocols at the same time #2378

  • Allow Inference Graphs to mix Protocols with the Executor #2299

  • Initial / immediate term base infrastructure for stateful metrics with feedback (custom metrics naming, concurrency coherence, etc) #2272

  • Extend seldon-single-model helm template to include metadata support #2070

  • Update ambassador doc and release doc about shadow model #1741

  • Remove OAuth code from Seldon Client #1677

  • Remove legacy Python 2 elements #1438

Merged pull requests:

v1.4.0 (2020-10-26)

Full Changelog

Fixed bugs:

  • protocols_example notebook is failing tests #2569

  • KEDA prom auto scale notebook is broken #2563

  • transport: is not respected on seldondeployment #2539

  • Add integration test for outlier detection server #2535

  • LibGL fix to be cherry picked for 1.3.1 #2534

  • ADServer Crashes due to updated dependencies #2533

  • Integration test failed due to alibi_explainer container #2529

  • Missing comma in the operator/config/manager/configmap.yaml #2526

  • alertmanager errors from prometheus #2525

  • Triton Server Image Incorrect #2511

  • Ensure all image errors are caugh on the build script #2509

  • Update helm publishing Makefile for seldon-core-kafka chart removal #2502

  • Seldon does not work with Gunicorn async workers #2499

  • CI build/push failures on tfserving-proxy image are not included in exit values #2477

  • Explicitly define default requests and limits for engine container #2475

  • Fix broken documentation links #1760

Closed issues:

  • add jsondata handling to req logger #2566

  • Seldon Core explainers to use alibi v0.5.5 #2562

  • Seldon Core 1.19 Kubernetes Support #2550

  • Add pidfile config for gunicorn #2546

  • Extend drift detector server (inside alibi detect server) to return metrics #2537

  • Update operator Redhat and OperatorHub integration #2532

  • Add requestPath back to the meta data #2531

  • easy to run out of disk with prometheus #2523

  • Support PDB specifications for SeldonDeployments #2508

  • Add KEDA support for seldon-core #2498

  • prometheus metrics for usage by seldondeployment #2483

  • Add SKLearn and XGBoost examples for MLServer / V2 Dataplane #2479

  • Support seldon-core running on knative serving like kfserving #2476

  • Triton server support with kfserving protocol #2460

  • Question about running grafana-prom for examples/kubeflow #2440

  • Data Science Metrics Core Update v1 #2397

  • configurable metrics port name for analytics #1809

Merged pull requests:

v1.3.0 (2020-09-29)

Full Changelog

Fixed bugs:

  • Unreadable notebook - sklearn_spacy_text_classifier_example.ipynb #2486

  • Alibi Detect Server libGL.so failed to find in image #2481

  • Seldon Docs failing on readthedocs #2455

  • 2 notebook tests failing #2454

  • SSL removed when executor multiplexing reverted #2447

  • JX master pipeline is failing to build and push images #2444

  • CI builds sklearn server with wrong sklearn version -> TestNotebooks.test_explainer_examples fails #2443

  • Install seldon-core-operator only working with old version #2438

  • Seldon Python Server memory leak in multithreading mode #2422

  • integration tests flakiness with TestPrepack.test_text_alibi_explainer #2408

  • integration tests fails: cannot import name ‘Turkey’ #2403

  • [doc] link to #Setup-Cluster is broken #2386

  • Unable to add more than one model in shadow deployment #2383

  • where is tfserving-mnist chart? #2372

  • semverComapre broken in some kubernetes flavours resulting in CRDs not being installed #2367

  • Revert multiplexing in the Executor #2364

  • Seldon pipeline crashes when there are a high volume of requests #2358

  • SeldonCore and random ‘upstream connect error or disconnect/reset before headersupstream connect error or disconnect/reset before headers’ errors on /predictions #2347

  • TerminationGracePeriodSeconds not respected in CRD #2332

  • remove trailing slash from graph metadata endpoint in docs #2322

  • Go version causes lint issues #2320

  • [doc] where is “seldon wrappers guideline”? #2307

  • Address flaky test test_model_template_app_rest_metrics_endpoint #2293

  • transform_output_raw not working . It gets refered to transform-input endpoint when analysing logs #2277

  • kfctl 0.5.1 is not available anymore #2258

  • Default user ID is always set to 8888 #2142

  • duplicate tensorflow_model_server command between entrypoint and args using prepackaged inference server #2133

  • Existing Webhook Secret Clashes if own certificate provided when doing upgrade #2101

  • can only join a child process #2094

  • Ensure all model servers have pinned requirements and the full requirements.txt is included in the docs #2065

  • curl response of the example sklearn_iris_jsondata is “Unknown data type returned as payload (must be list or np array):None” #2063

  • upgrades briefly go to a Failed state before Available but work the whole time #2044

  • SeldonDeployment with just a shadow is allowed past validation #2022

  • Seldon Core operator Crashes when deployment with empty predictor is passed #2020

  • OpenAPI Validation for PredictiveUnits limited to 5 levels #1864

  • seldon-core-operator CRD’s incompatible with K8s v1.18 #1675

Closed issues:

  • Add selectorpath for /scale subresource in SeldonDeployment CRD #2485

  • Dependabot can’t resolve your Go dependency files #2464

  • Question about seldon-controller-manager setup on kubeadm #2452

  • Dependabot can’t resolve your Go dependency files #2445

  • Add Concepts page to Seldon Core docs #2433

  • option to scrape prometheus less often #2401

  • Add Flag in the routing protocol to skip further processing and return #2400

  • Using ArgoCD to deploy Seldon-core-operator shows Webhooks as OutOfSync #2392

  • Add link to alibi notebooks to reference how the explainer models are built #2371

  • complex graph only expose endpoint for last metioned container and does not pass forward the output of the parent model #2370

  • Support YAML for SeldonDeployment definition in examples under seldon-core/helm-charts #2362

  • Add TreeShap Explainer example #2361

  • Update operator & executor k8s libraries to 1.18 (or 1.19) #2360

  • Inference Graph Example #2331

  • Setting and using SELDON_ENVIRONMENT for Request Logging to use one ELK Cluster for multiple Environment #2328

  • Setting and using SELDON_ENVIRONMENT for Request Logging to use one ELK Cluster for multiple Environment #2327

  • extend metadata schema to provide a field for custom entries #2312

  • Remove pytest-runner dependency from setup.py #2303

  • Remove depedency on deprecated pytest-runner #2302

  • Authentication support for ELK #2298

  • Add documentation and example for feedback reward leveraging custom metrics #2271

  • publish 1.2.2 RedHat operator #2244

  • MLFlow Model on MinIO Not Loading #2213

  • helm charts documentation #2203

  • Add / Extend docs on seldon-core-microservice #2202

  • Add “seldon.io” prefix path to all kubernetes labels associated with Seldon #2187

  • Change docker build context for executor to speed up build process #2186

  • Upgrade Alibi Integrations #2160

  • SeldonDeployment explainer description #2144

  • Remove storage.py from python module #2140

  • Refactor logging in Executor #2090

  • Make the helm chart generator part of the release script #2072

  • Upgrade k8s client API to 1.18+ #1949

  • silence flask logs from prometheus probing python wrapper #1907

  • Update SeldonDeployment Helm charts #1879

  • Grafana Dashboard not updating the deployments #1854

  • Hyphens in names cause the service orchestrator to start a grpc server #1850

  • SKLearn version support too low #1813

  • Seldon core wrapper support for Spring 2 #1796

  • Align GPU TF Python Image requirements and structure #1789

  • Investigate test_model_template_app_grpc_metrics flakiness #1745

  • support runtime request tags / metrics in thread/process safe way #1735

  • Support NVIDIA/KFServing V2 Data Plane #1648

  • Multiplex Http and gRPC traffic #1628

  • Update Kubeflow Example to 1.0 #1509

  • azure-storage-blob package update to 12.1.0 #1371

  • DownwardAPI fails validation in CRD #926

  • Update CRD to be Structural #641

Merged pull requests:

v1.2.3 (2020-08-14)

Full Changelog

Fixed bugs:

  • Shadow model gets no traffic #2225

  • kubeflow central dashboard we could not see Manage Contributor menu #2223

  • Tensorflow session hangs in gunicorn worker process #2220

  • seldon operator giving error #2184

  • Python licenses change depending on the environment #2124

Closed issues:

  • Add source mirroring for MPL licensed dependencies #2263

  • Is it possible to pass init parameters to Predictor class through seldon-core-microservice #2250

  • How to solve race conditions between two requests. #2240

  • Update to use KFserving 0.4.0 artifacts #2236

  • Add CVE checks as part of CI #2183

  • GCP Marketplace Release Update #1804

  • Add kubernetes labels to help with selectors #1405

Merged pull requests:

v1.2.2 (2020-07-28)

Full Changelog

Fixed bugs:

  • Alibi Detect Drift does not use batch #2194

  • Explainers are hardwired to seldon protocol #2185

  • Address build stability #2175

  • Seldon-batch-processor Issue #2173

  • Jenkins X Pipelines are not marked as finished #2148

  • Robustness of operator_upgrade notebook #2119

  • Unable to view feedback reward on Grafana dashboard #2115

  • MLFlowServer predict function ignores feature_names parameter #2113

  • Request logger drops incoming requests for traffic coming from a single model #2109

  • deep_mnist example: failed calling webhook “v1alpha2.mseldondeployment.kb.io” #2107

  • Duplicated Mutating Webhooks can Coexist without Notice #2103

  • READWRITEMANY does not work on GCP #2102

  • There is no spam.csv in dir examples/input_tranformer, the example input_tranformer doesn’t work #2087

  • seldon-controller-manager crashing #2066

  • No module named ‘sklearn.linear_model._logistic’ when using the docker image seldonio/sklearnserver_rest:1.2.0 due to scikit-learn==0.20.3 #2059

  • python: Relink error in GPU image #2048

  • Seldon Azure Deep Mnist tutorial CrashLoopBackOff while creating pods #2043

  • Address flakiness of batch processing integration test #1985

  • “Empty Json Parameter in data” for model components in Spam Classifier Example #1938

  • ambassador helm chart deprecation warnings #1928

  • Tags created by components inside combiner don’t propagate #1927

  • Duplicate ports defined in seldon-container-engine container #1799

  • idletimeout between envoy and executor #1797

  • Helm Chart - Seldon Core Analytics - extraEnv and VirtualService not working anymore. #1791

Closed issues:

  • CNCF-Runtime discussion/presentation(?) #2181

  • Make Azure dependency optional #2168

  • How to pass a contract.json as curl request. I keep getting bad data when i send a contract.json file as curl. #2151

  • Serving local (host) model with the prepackaged TensorFlow server #2146

  • Update version of Jaeger in Python wrapper #2143

  • Add to docs clarification on Routing not available in executor #2139

  • Upgrade k8s.io dependencies in the Executor #2134

  • Upgrade knative.dev deps in Operator #2128

  • bump zap from v1.10.0 to v1.15.0 #2127

  • upgrade istio.io dependencies in operator #2126

  • make mock-classifier a RELATED_IMAGE for redhat operator #2118

  • Upgrade controller-runtime in Operator #2116

  • Upgrade Operator dependencies that can be bumped without problems #2098

  • Remove Executor’s dependency on client-go #2092

  • Upgrade Operator version in Executor deps #2091

  • Upgrade Executor dependencies that can be bumped without problems #2089

  • Update dependencies of Operator and Executor #2088

  • request logger retries #2079

  • allow loading wrapped model from installed package #2068

  • Is there a way to specify URL of swagger-ui static resources instead of https://cdnsjs.cloudflare.com in a intranet k8s cluster #2067

  • Determine 1st and 2nd Dependencies for Go operator and executor #2061

  • support multiple named tensors in seldon protocol and seldon-core client #2049

  • Drop podinfo volume name backwards compatibility transition in 1.3 release #2024

  • Return pointer instead of value in SeldonApiClient methods #2014

  • Enable production mode in Python server by default #1993

  • update UPGRADING.md with new name of rolling image #1989

  • re-define noEngine annotation #1976

  • Ability to return all outputs from tensorflow serving grpc #1965

  • Allow to specify model version for tensorflow serving #1964

  • Automate license check in CI linting pipeline #1932

  • pass ServiceAccountName in predictor to prepackaged servers initContainer #1865

  • Using “required” field for key values in helm chart #1784

  • Update Ambassador Circuit Breaker Example to have parallel requests #1753

  • Enable production settings in loggers #1737

  • Create a benchmarking framework #1731

  • GRPC Auth problem with GCP IAP #1719

  • Serialization of pre-processing pipeline for CI/CD #1713

  • Add and example Notebook for Istio Setup and Integrations #1712

  • Seldon Build Permission Denied #1689

  • Autogenerate an OpenAPI spec and SDK #1682

  • GPU deadlock for pytorch models using the python wrapper #1662

  • Convert Request Logger Example Scripts into Helm Chart #1511

  • Flask Theading bug, when using with sockeye and mxnet #1498

  • Improve release notes #1471

  • Migrate tutorials to use kind instead of Minikube #1256

Merged pull requests:

v1.2.1 (2020-07-01)

Full Changelog

Fixed bugs:

  • upgrading from 1.1 to 1.2 with existing sdep leads to volume mount error #2017

  • Seldon Operator automatic update from v1.1.0 to v1.2.0 causes seldondeployment failures #2009

  • Changing predictor.replicas causes all deployment pods to be replaced #2008

  • requests per second from seldon_api_executor_cient_requests_count not right? #2004

  • Cannot connect to metrics port 6000 for custom models wrapped with s2i. #1988

  • Helm switch rbac.create=false does not fully work #1982

  • serviceAccount.name not used in Helm chart templates #1977

  • executor service not targeting to executor #1975

  • Issue in “seldon-container-engine” with MLFLOW_SERVER #1922

  • Permission denied while reading ./openapi/seldon.json in seldon-container-engine #1855

  • Allow custom pip dependencies in MLFLOW_SERVER #1547

  • Stop integration tests if setup fails #1417

Closed issues:

  • upgrading notes for 1.2.1 #2051

  • Dependabot can’t resolve your Go dependency files #2003

  • Set executor port from model deployment file #1974

  • Readiness probe failed seldon-container-engine while deploying the pipeline #1963

  • Potential dependency conflicts between seldon-core and opentracing #1867

  • Add documentation that explains how to configure the named port #1853

  • Built-in header based routing #1739

  • Update release process #1732

  • Include prepackeged servers in GA and version using version.txt #1726

  • Can’t create resources under v1.18 #1678

  • Use zap.New instead of zap.Logger #1657

  • Wrong package for YAML #1609

  • Batch Processing Exploration for Seldon Core #1413

  • Offline Batch Integration #1391

  • Add Health and Status endpoints to grpc spec #1387

Merged pull requests:

v1.2.0 (2020-06-18)

Full Changelog

Implemented enhancements:

  • request logger to support bearer token auth #1773

  • Feature request: Provide a metadata endpoint with custom metadata #319

Fixed bugs:

  • Golang Service Orchestrator (Executor) fails to replace variables since no longer runs root #1955

  • seldon-single-model chart doesn’t install #1942

  • How to access rolling accuracy metrics? #1926

  • curl command on seldon core overview doesn’t work #1918

  • can’t change analytics nodeExporter port #1916

  • Empty json parameter in data #1912

  • Multipart form data not passed between chain of seldon components #1899

  • Integration tests (test_mlflow) failing on Master #1896

  • Metrics notebook broken (gRPC) #1886

  • Helm chart seldon-manager role missing permissions for createResources: true #1885

  • Document Mistake about Endpoint Testing #1881

  • Routers not working with new Executor #1876

  • Notebook test failures #1869

  • Grafana Dashboard #1840

  • Publish new core builder image #1828

  • Feedback method is not handled in the engine’s REST API #1825

  • Cant use the endpoint of kubernetes service created by the model #1824

  • Logs don’t have information of which predictor it comes from #1817

  • Download specific Helm version in core-builder/Dockerfile #1811

  • Fluentd Helm Chart update breaks logs integration #1780

  • Seldon Core v1.1.0 helm chart has outdated seldon-config configmap #1779

  • Metrics executor local sample REST request fails #1770

  • Add XSS patches to executor #1766

  • Executor sample gRPC local loggger fails #1763

  • model server pods not created if controllerId set? #1758

  • Can’t have a colon in the registry name with the Seldon-Core Python Package #1756

  • failed calling webhook #1742

  • Executor container not exposing core metrics #1729

  • problem configuring S3 access from MLFlow server #1727

  • seldon.io/rest-read-timeout for ambassador doesn’t take effect in executor 1.1.0? #1724

  • createResources flag in helm chart looks backwards #1723

  • Incorrect seldon-core package version in 1.1.0 builder image #1718

  • custom tags in release 1.1.0 not working #1715

  • Scrape related annotations only in deployment and not in services caused prometheus to ignore seldon analytics metric endpoints #1705

  • Update swagger to work in executor and be up to date #1703

  • web client metrics not working #1685

  • com.google.protobuf.InvalidProtocolBufferException: com.google.gson.stream.MalformedJsonException: Unterminated array at line 1 column 32772 path $.data.ndarray[1] at io.seldon.engine.pb.JsonFormat$ParserImpl.merge(JsonFormat.java:1132) ~[classes!/:0.5.1-SNAPSHOT] at io.seldon.engine.pb.JsonFormat$Parser.merge(JsonFormat.java:313) #1680

  • Unable to use HPA with a pre-packaged “implementation:TENSORFLOW_SERVER” model due to “tfserving” container not having requests/limits defined #1676

  • Executor build failures #1663

  • Java wrapper do not accept application/json requests #1603

  • Images created with s2i don’t work as explainers when added as containerspec #1549

  • Engine Thread Pool Configuration #1490

  • test_no_permission_bucket_gcs fails when run locally #1364

  • Benchmarking notebook fails with “Deployment not found” error #1177

Closed issues:

  • From Kubeflow to Seldon Documentation #1961

  • Update Payload Logger to use V1 Cloudevents #1958

  • Make Operator and Executor Dockefile executable dynamically linked #1951

  • Notebook Test Updates #1948

  • Update Licenses for 1.2 Release #1947

  • add managed-by label #1936

  • extend metadata to cover seldon message #1933

  • Seldon-core-microservice-tester command available for ambassador endpoint #1914

  • seldon-core-manager pod is getting restarted #1910

  • /metrics endpoint to return a json file #1905

  • Is /metrics endpoint accessible #1901

  • How seldon custom metrics work? Can it be seen on grafana? #1898

  • Ensure Upgrading to 1.2.0 complete #1888

  • Multiplex HTTP and gRPC on the operator #1887

  • Add back Java engine’s integration tests #1878

  • option for request logging to go to a single namespace #1859

  • track cloudevents version #1857

  • Ensure all documentation links are relative #1851

  • update model metadata notebook example #1829

  • Kind test script Istio install failing with kubectl version v1.18.x #1802

  • Better names for metric ports of SeldonDeployment containers #1798

  • Create new Release PIpeline in Jenkins X that works on separate branch #1793

  • To pass customized input json without the mandatory jsonData parameter #1792

  • Add a section in the python docs to talk about environment.yaml for conda #1788

  • Get timeout error on seldon-container-engine if I set GUNICORN_WORKERS env #1777

  • Add explainer image version to configmap #1776

  • Allow access to headers in python wrapper #1769

  • Multiplex HTTP and gRPC on the executor #1762

  • GCP Marketplace Update CRD Create #1755

  • docs: have a single notebook that helm install minio #1740

  • RedHat version of core servers #1734

  • Pachyderm integration #1733

  • provide a graph-level model metadata #1728

  • RedHat Operator Updates for 1.1.0 #1710

  • Add docs for istio annotations #1707

  • log seldon-core and wrapper version after microservice start #1704

  • Running release script will remove chart comments #1690

  • add model metadata support #1638

  • allow to use Java engine if helm installation defaults to executor #1607

  • ensure we’re using open source ambassador in examples #1581

  • High number of concurrent users causing Seldon API to timeout #1558

  • Add support to add ambassador circuit-breaker in seldon deployment #1556

  • Move notebook tests to separate test command #1538

  • com.google.protobuf.InvalidProtocolBufferException: null #1532

  • ENGINE_CONTAINER_SERVICE_ACCOUNT_NAME and EXECUTOR_CONTAINER_SERVICE_ACCOUNT_NAME is picking a default value #1508

  • UBI version of the latest Seldon operator image #1482

  • Request metadata: discussion and design #1480

  • Send back PUID on response #1460

  • Support custom client protobufs in grpc server #1420

  • Alibi:Detect Outlier Detection Integration #1393

  • Define future approach to metadata and metrics with new Executor #1362

  • Update outlier detection examples #1288

  • make ambassador shadow consistent with istio #1109

  • can we expose health check api in internal python service similar to what we have in external like /ping #770

  • Decouple tensorflow package from the base python seldon-core image. #564

  • intermittent UNIMPLEMENTED error from ambassador #473

  • conda-forge integration #341

  • Profile the CRD operator #167

Merged pull requests:

v1.1.0 (2020-04-16)

Full Changelog

Fixed bugs:

  • Client Go library for istio not backwards compatible #1695

  • Change executor timeouts #1691

  • Should only set istio route weight when seldon traffic field value > 0 #1658

  • [document] Seldon Core Roadmap is missing #1646

  • Clean up of VirtualServices active for non-istio scenarios #1631

  • Explainer prediction errors when spec-name or predictior-name change #1626

  • Explainer example notebook use v1alpha2 CRDS #1622

  • cifar10 custom logger not recording dataType #1617

  • wrapper compatibility table not visible in sphinx-rendered docs #1612

  • url with tensorflow protocol #1611

  • Ensure SNAPSHOT builds contain unique identified (hash, date, etc) #1605

  • PrePackaged Server deployment failure #1599

  • Duplicated VirtualServices entries #1594

  • Empty explainers on old deployments #1593

  • Inference graph becomes unavalible with no error in logs #1584

  • notebook links dead in mdincluded markdown files #1583

  • Changing names of componentSpecs causes reconcile error #1571

  • Different behaviors between seldon-engine v0.4.1 and v1.0.2 #1569

  • seldon-core-crd helm chart is out of date #1563

  • Add labels from componentSpecs.metadata to dpeloyments #1561

  • Istio creates different path to Ambassador for explainer #1548

  • Deployments created by Seldon are apps/v1 but HPA scaleTargetRef is extensions/v1beta1 #1533

  • Invalid JSON: Unexpected type for Value message when passing tuples in json_data #1527

  • Engine has multiple Makefiles and Dockerfiles #1521

  • Reduce flakiness on rolling update tests #1513

  • minio example #1506

  • missing python dependencies for integration tests #1504

  • Error in SeldonClient in debugger #1499

  • Jenkins X integration broken #1484

  • executor: tags do not propagate through inference graph #1474

  • seldonio/seldon-core-executor image versions need updating #1469

  • Seldon Core Deployment Istio virtualservice is only created until after the pod is ready resulting in service unavailable until it propagates #1461

  • Predictive Unit/Deployment Environment Variables not set by operator #1449

  • Errors in seldon_core_setup notebook #1440

  • Errors in helm_examples notebook #1436

  • Errors in explainer_examples notebook #1434

  • Executor version not updated in Kustomize during release #1432

  • Seldon controller crashes when submitting deployer #1418

  • Flakiness on integration tests #1402

  • operator/helm/create_templates.sh generated files can have order of sections different on different machines #1291

Closed issues:

  • logger to identify single-element numeric reqs or responses as ‘number’ #1699

  • Add Outlier and Drift examples to Seldon Core #1692

  • Create Alibi Detect Server #1667

  • Request for Java version of Seldon Grpc Client #1656

  • standardise requests in req logger #1644

  • SeldonClient doesn’t currently work with explainers on gateway #1627

  • API group should be “machinelearning.seldon.io” not “machinelearning” #1615

  • Remove old docs folder #1610

  • Clarify for how long Java engine orchestrator will be supported #1608

  • Document s2i wrapper compatibility with Seldon Core 1.1 #1601

  • Be able to scale SeldonDeployment objects using kubectl #1598

  • Default Engine Java Opts exposes Vulnerability #1597

  • [document request] Add more technical details about service orchestrator #1596

  • Inquiry - Java JMX Server Security Vulnerability #1595

  • Add Scale SubResource to CRD #1592

  • Make inclusion of metrics in SeldonMessage configurable in 1.1 #1582

  • Passing status code back from executor service orchestrator #1574

  • Separate Jupyter Notebook Tests from E2E tests to improve speed of tests #1567

  • Avoid logging that custom_tags or custom_metrics is not implemented on every request #1565

  • Document compatbility of wrapper versions with seldon engine versions #1564

  • Surface reconcile errors as Events on the SeldonDeployment #1560

  • New protocol and transport fields should be moved to higher level in CRD #1552

  • Upgrade process from v1.0.2 #1551

  • Upgrade Blogs Section in Documentation #1550

  • seldon-core-analytics should not use hostPort #1539

  • The variable envSecretRefName currently has to be set up on a deployment level but would be required to have global default for seldon-core #1530

  • OpenShift 4.3 compatibility for Seldon-Operator 0.1.5 #1524

  • Add doc for all images used by Core #1515

  • request logging prominence in docs #1514

  • Change TFServing image in helm chart to point to specific version instead of latest #1512

  • Add top level registry prefix to Helm Charts for custom registries #1510

  • Allow for default Seldon Core request logger endpoint to be set on helm chart #1501

  • seldon support muti istio gateway #1500

  • Change explainer type to pointer in go struct #1485

  • Prediction API should define a dedicated image type #1478

  • Support Seldon Core v1 in Operator Framework Community Operator #1477

  • Expose /metrics prometheus endpoint in python wrapper #1476

  • Add breaking changes for 1.1 to docs #1475

  • Clean up examples #1467

  • failed calling webhook “mutating-create-update-seldondeployment.seldon.io”: error #1462

  • Clean up notebook examples #1456

  • Remove benchmark_simple_model notebook #1455

  • executor tracing enhancement #1444

  • Sync s2i wrapper image versions with Seldon Core releases #1439

  • Ambassador setup fails to complete on minikube #1435

  • Seldon v1.0 CRD uses new (k8s 1.15) feature that not backward compatible with k8s versions (<= 1.14) #1431

  • Cherrypick PRs for 1.0.2 Release #1429

  • Document custom pre-packaged model servers #1416

  • new v1.0.0 CRD’s resources validator is not backward compatible #1410

  • Support for Xgboost 0.9 #1394

  • Updated Seldon Data Plane #1389

  • Add annotation support for executor #1384

  • Possible enhance ambassador customization in seldon deployment #1378

  • Seldon controller pod crashes on deploy when no internet connection available #1374

  • request logging for shadows #1207

  • Integration tests for tracing #977

  • Custom errors raised in Python microservice don’t make it back to the client #939

  • Prometheus nodes get evicted, limits could be added to avoid this #934

  • Scaling of SeldonSeployments #884

  • tensorflow-gpu #602

  • Reduce size of images created by Python wrapper #312

  • gRPC metrics not exposed to prometheus #302

Merged pull requests:

v1.0.2 (2020-02-17)

Full Changelog

Fixed bugs:

  • Explainer not garbage collected if spec updated #1427

  • Tensorflow REST example using inputs fail #1421

  • Missing binaries for grpcio 1.27 #1409

  • Customization of seldon ambassador url #1401

  • Import error for seldon_core module in python3 #1396

  • v1.0.0 Seldon operator crashes when fail to deploy sdep. #1385

  • Seldon Core Webhook Fails Randomly #1361

  • Segmentation violation in seldon operator #1359

  • module does not work once deployed to kubernetes #1345

  • Remove redundant travis build badges #1343

  • Namespaced operator duplicates Seldon deployments #1339

  • update redhat operator #1327

  • Internal build issue: PR builds intermittently fail with no space left on device #1322

  • ensure we have resource metrics in analytics #1317

  • hpa cannot work #1310

  • Existing resource conflict with Helm 3.x #1289

  • CVE-2019-18224 #1286

  • Update analytics docs with Helm v3 #1265

  • broken links to image descriptions #1249

  • Tox tests fail if py37 is not present #1247

  • Shadow example fails in Istio notebook #1179

Closed issues:

  • Update kustomize default resources for manager #1412

  • Add kubeflow install requirements #1411

  • helm value settings to configure seldon-controller-manager resources #1404

  • Wrong model format error when tried serving an xgboost trained model with seldon-core #1403

  • New Service Orchestrator #1388

  • Helm charts updates needed from kustomize on master #1375

  • replace print with logging in tests #1372

  • Add 3rd party licences to Docker images #1360

  • Ensure no downtime during operator updates #1354

  • How to craft an ambassador-config override with shadow mode #1347

  • Update InitContainer Downloader Image to latest kfserving containing s3 fix #1341

  • Seldon Core Operator fails to install on EKS #1320

  • Update Analytics Helm Chart #1297

  • Add operator tests to Jenkins X #1295

  • Update request logger #1293

  • Kubeflow 1.0 integration (Updating kubeflow repo to have latest Seldon 1.0) #1284

  • Internal REST based encoding schemes could be improved #1270

  • Default JVM heap max size (-Xmx) should be increased #1269

  • provide a docker image ready to run example notebooks? #1235

  • HPA does not work with RollingUpdate #1226

  • canary flag like shadow flag #1210

  • guidelines on annotations for metadata #1208

  • Seldon core analytics should scrape resource metrics #1206

  • google storage slowness - initializer hangs in PodInitializing #1204

  • snapshot helm chart not being published? #1203

  • Move inference servers and clients to root folder #1198

  • Wrong metrics service port #1197

  • resources for integration tests #1194

  • Cannot set GRPC Max message size annotation: ValueError #1165

  • Add run-linter PROW command via Jenkins X #1151

  • Engine failing to access /tmp folder when restricted Pod Policies in place. #1143

  • Merge Conda and regular Python s2i wrappers #1125

  • Explore alternatives to protobufs’s Struct and Value #1119

  • Ensure only changing weight of predictor doesn’t cause new rollout #1078

  • switching between application/x-www-form-urlencoded and application/json #1013

  • Empty Children array on a Seldon Deployment causes ArgoCD sync issues #1009

  • Forward headers that orchestrator gets to containers in the graph #967

  • Seldon Operator with Istio enabled does not work with Strict mTLS. #899

  • Custom timeouts might not be used with REST api #874

  • Prometheus endpoint sending invalid type for Histograms #740

  • tracing not sharing context from ambassador? #731

  • Deprecate s2i-python2 in examples #593

  • Allow route method to use metadata #591

  • Clearer usage/defaults or warnings about GRPC components in the deployment graph #553

  • Custom metrics in feedback #315

v1.0.1 (2020-01-15)

Full Changelog

Fixed bugs:

  • operator CI build fails #1330

  • Remove log4j from H2O example #1318

  • cert-manager version #1262

Closed issues:

  • Old pods not removed after rolling update #1325

  • seldonio/seldon-core-s2i-python3-tf-gpu:0.15 image default python version is 2.7.17 #1324

  • send_feedback() got an unexpected keyword argument ‘routing’ #1321

  • Helm 2 install fails for v1.0 #1299

  • Clean up API types and Webhooks #1294

  • Include seldon proto compilation in GO #1245

  • helm upgrade when the operator configmap has changed? #1135

  • End-to-end tests for Pre-packaged model servers hang if name doesn’t match exactly #820

  • S2I builder images should have pinned Python dependencies #340

Merged pull requests:

v1.0.0 (2019-12-18)

Full Changelog

Fixed bugs:

  • Certificate not added for all CRD versions in helm chart #1275

  • Webhook not updated for multi version handling #1274

  • Ambassador 404s on Canary Test #1271

  • Delete hpaSepc in seldon deployment doesn’t delete the hpa object #1263

  • CVE-2019-5482 and CVE-2019-18224 #1261

  • ambassador version #1260

  • No module named ‘tensorflow.examples.tutorials’ with tensorflow 2.0 #1248

  • SeldonDeployment stuck on creating when an environment variable is a reference #1211

  • cat.json missing from Explainer notebook #1178

  • Analytics charts broken in K8s v1.16 #1176

  • Helm install issues on k8s 1.16.2 #1095

  • Model initializer does not work with S3 #885

Closed issues:

  • Namespace operator e2e tests #1281

  • Istio integration requires sidecar injection #1273

  • Serialization exception when using the internal prediction API endpoint #1241

  • seldon-core 1.0 release prep #1239

  • Webhook Selectors only available in k8s &gt;= 1.15 #1233

  • duplication of seldon_core_setup notebook #1232

  • crd issue on 1.16 #1225

  • Centralised logging not working on k8s 1.16 and helm 3 #1224

  • Port binding in Java wrapper clashes with the engine #1223

  • uuids in request logs #1209

  • User “system:serviceaccount:kubeflow:pipeline-runner” cannot create resource “seldondeployments” #1205

  • Add PR comment for build pipeline #1200

  • Fix docs install reference to 0.5.0 #1196

  • Unschedulable: pod has unbound immediate PersistentVolumeClaims #1191

  • Text batch data is not split into multiple requests by request logger #1189

  • latest snapshot not working in GKE #1173

  • local e2e don’t build operator image #1171

  • e2e tests broken in master #1164

  • Move Go wrapper to incubating #1157

  • Update examples in line with Helm v3 #1154

  • Update Install docs in line with Helm v3 #1153

  • Add triage label to new issues #1152

  • add prepackaged model server pvc example #1150

  • running end to end tests on local machine #1147

  • Can we attach pvc to model-initializer(storage-initializer) ? #1146

  • Standardise Data Mappings in Seldon Wrapper #1145

  • Wrong types in Seldon Core user methods #1144

  • Create operator client-set #1141

  • Investigate impact of helm v3 in seldon-core #1140

  • Seldon Core Operator Restricted to Single Namespace #1139

  • Need to update integration test script with helm 3.0 version #1138

  • Can not control retry_policy from SeldonDeployment yaml file #1137

  • [Improvement] Obscure service name when deploy my yaml #1128

  • Jenkins X currently creates a new changelog tag / version every time a PR is landed #1124

  • strData requests are not printed by seldon request logger #1121

  • adding canary, shadow or explainer shouldn’t affect main predictor #1110

  • Unable to mount model from PVC into tf serving prepackaged model server #1106

  • Remove Travis Integration #1105

  • Move CRD to v1 #1100

  • Upgrade Maven and JDK on CI image #1094

  • Fix sporadic failures with e2e tests #1084

  • Inconsistent return value for explain method in SeldonClient #1083

  • Refactoring handling httpResponse in Java Engine #1075

  • following tutorial got 503 #1073

  • Validation fails if componentSpecs.metadata.creationTimestamp is not specified #1061

  • Move non 1.0 components to incubating folders #990

  • Improve PrePackaged Model Servers #959

  • use fixed version for model initialiser image #957

  • Helm Upgrade Process #890

  • Dynamic Engine version support at Seldon operator #871

  • Integrate GPU Seldon Core Image into Build Scripts #868

  • Custom prepackaged model servers #857

  • Feature request: Python Seldon Client: support sending gRPC data with meta field #821

  • Allow access to puid within the predict API #795

  • Installing seldon-core-operator requires clusterwide RBAC and should be installed by a cluster admin #670

  • Update notebooks to refer to seldon install rather than include code #646

Merged pull requests:

v0.5.1 (2019-11-21)

Full Changelog

Fixed bugs:

  • MLFLOW_SERVER ModuleNotFoundError #828

Closed issues:

  • Engine using separate pod, ignoring annotations #1120

  • How to pass ModelInitializerContainerImage path #1116

  • how to define MyModel for PyTorch model which has more than one arguments? #1115

  • how can I wrap a PyTorch Image with customed network? #1114

  • Cannot create a SeldonDeployment having volumes of type projected and configMap. #1072

Merged pull requests:

v0.5.13 (2019-11-15)

Full Changelog

Closed issues:

  • Move API path from v0.1 to v1.0 #991

Merged pull requests:

v0.5.12 (2019-11-15)

Full Changelog

v0.5.11 (2019-11-15)

Full Changelog

Closed issues:

  • Shadow deployment for Istio #741

Merged pull requests:

v0.5.10 (2019-11-15)

Full Changelog

Fixed bugs:

  • Operator crash if one container in pod not created properly #1104

Merged pull requests:

v0.5.9 (2019-11-15)

Full Changelog

Closed issues:

  • Modify engine’s Proto Value to JSON conversion to avoid int-to-float conversions in REST requests #948

v0.5.8 (2019-11-15)

Full Changelog

Merged pull requests:

v0.5.7 (2019-11-15)

Full Changelog

Closed issues:

  • Update Python SeldonClient to handle JSON payloads #607

Merged pull requests:

  • Update Python SeldonClient to handle JSON payloads #1082 (axsaucedo)

v0.5.6 (2019-11-14)

Full Changelog

Merged pull requests:

  • Add /health/ping and /health/status endpoint to Python REST Microservices #1026 (kparaju)

v0.5.5 (2019-11-14)

Full Changelog

Merged pull requests:

v0.5.4 (2019-11-14)

Full Changelog

Closed issues:

  • Improve speed of execution for integration tests #1032

Merged pull requests:

v0.5.3 (2019-11-14)

Full Changelog

Security fixes:

Closed issues:

  • Seldon AB testing - getting an error “info”: “Parameter ‘ratioA’ is missing.” #1081

  • remove old CRD generation #1079

  • Broken integration tests #1076

  • DOCKER_IMAGE_VERSION in Makefile is only 0.3 and 0.4; public docker images go up to 0.13 #1071

  • SeldonDeployment modelUri support more protocol #1070

  • Your .dependabot/config.yml contained invalid details #1045

  • Add tox #1042

  • Broken imports on 0.5.0.2 release #1040

  • Integration tests fail intermittently #1031

  • Major version changes on dependencies could cause issues #1029

  • with the latest release of azure-storage-blob 12.0.0, seldon build fails #1027

  • Extend documentation to include optional dependencies and azure blob quickfix #1025

  • Update release.py script to change version in kustomization.yaml #1024

  • Set upper limit for azure-storage-blob version #1023

  • Make GCS support optional #1018

  • Update Talks/Blogs/Videos/Use cases in Docs #1017

  • No available release name found for Seldon-Core-Operator #1014

  • Fully automate CI/CD process and introduce manual trigger for release process #986

  • Use official Helm charts for Grafana and Prometheus #965

  • Set JSON_SORT_KEYS and JSONIFY_PRETTYPRINT_REGULAR to False by default #964

  • Enable Storage.py to be able to download from HDFS URL #963

  • Fix helm version in Chart.yaml #961

  • Code style standardisation for Seldon Core Python modules #947

  • Automated Test and Build Hooks #933

  • Seldon Image at “seldonio/seldon-core-s2i-python3-tf-gpu:0.12-SNAPSHOT” not able to find GPU devices #914

  • Update Java dependencies #902

  • Create 1.0 GA Document #887

  • Enable Storage.py to be able to download from URL file #883

  • Seldon container engine has not resource limit #769

  • Java Seldon Engine doesn’t pass microservice HTTP exceptions upstream #705

  • Feature_names is redundant when using jsonData in predict. Also meta is redundant in response, as meta can be combined in the jsonData. #665

  • Modify helm chart seldon-core-analytics using chart dependencies #613

  • expand contributing guide #569

  • requirements.txt for Python wrapped models should be a configurable name/path #548

  • Docker Image is too Big #526

  • Deployment of seldon as a new custom resource via fabric8 #486

  • Update developer docs #203

Merged pull requests:

v0.5.0 (2019-11-01)

Full Changelog

Closed issues:

  • Update logback #1007

  • seldon-core-operator fails to install on Kubernetes 1.16 #1004

  • Custom error raised in Python model was not passed back to the client #974

  • Models with multiple input types are not supported #921

Merged pull requests:

  • Removed the hash as it was crashing the command when the pipeline was ran #1022 (axsaucedo)

  • Add pre-commit hook for black and fix linter #1020 (adriangonz)

  • Added documentation on how to support Models with multiple input types in python wrapper #1015 (axsaucedo)

  • Move from logback to log4j2 #1008 (adriangonz)

  • Adding functionality for running e2e tests in Jenkins X #994 (axsaucedo)

v0.4.5 (2019-10-31)

Full Changelog

Closed issues:

  • Update transitive dependencies #1002

  • /helm-charts/seldon-core not present in seldon-core repository for CICD demo #970

Merged pull requests:

v0.4.4 (2019-10-30)

Full Changelog

Closed issues:

  • serviceAccountName has no effect for pre-packaged implementation #1012

  • Model version duplication when rollout times out #916

Merged pull requests:

  • Customized error raised in Python model needs be passed back to client #1010 (lennon310)

v0.4.3 (2019-10-29)

Full Changelog

Closed issues:

  • Update deployments to apps/v1 for 1.16 k8s #1005

Merged pull requests:

v0.4.2 (2019-10-28)

Full Changelog

Merged pull requests:

v- (2019-10-27)

Full Changelog

Closed issues:

  • Update Python dependencies #987

  • SvcOrch resources overwritten #982

  • Security vulnerabilities on Java dependencies #981

  • Engine doesn’t push traces #975

  • Make Helm chart repository available on Helm Hub #973

  • huge data set #972

  • Seldon deployment issue when adding initContainer #968

  • Remove Kustomize folder #960

  • Explainer args causes reconcile loop in manager #954

  • unable to create SeldonDeployment #952

  • Make it possible to override seldon prediction service’s service name #950

  • Make it possible to override seldon prediction service’s service name #949

  • invalid json should not be a 500 #943

  • Tidy up Java project #937

  • helm-charts/seldon-core does not have seldon-core #936

  • Why seldon chose application/x-www-form-urlencoded as Content-Type instead of application/json when treating predict requests? #935

  • gcp private bucket instructions #927

  • seldon operator resource name changed #919

  • controller now deployment not statefulset #917

  • Best practice for multiple servings from one model #915

  • Seldon Controller Manager CrashLoopBackoff #912

  • Allow –max-requests and –max-requests jitter parameters for python wrapper #911

  • Feedback API is returning empty string, how send_feedback works ? #910

  • Update python wrapper to pass bytes instead of string #909

  • Python wrapper 0.13-SNAPSHOT needs updated setuptools in setup.py #908

  • Engine not catching grpc exceptions cleanly #907

  • Python Wrapper not exposing all Services via grpc #906

  • Seldon Client is not passing binary data field properly with REST #897

  • Building engine on macOS #894

  • Block content type sniffing #892

  • Plan to unify external and internal API paths #889

  • Build automation for 1.0 Plan #888

  • Remove Redis from CICD scripts #882

  • Deploy overview image doesn’t appear in Seldon deployment of income classifier and Alibi anchor explainer example #880

  • Top navigation bar doesn’t use Seldon’s pink (#ff4072), and README uses wrong form of “lets” #878

  • Internal error occurred: failed calling webhook #877

  • Rest request doesn’t give the expected answer while using the predict function #876

  • Requesting the endpoint only works the first time after the endpoint is available, the subsequent requests give “AttributeError: ‘Logic’ object has no attribute ‘predict_raw’” #860

  • Add post-commit hook for s2i builder images #852

  • Should seldondeployment generate stable deployment name? #851

  • Allow operator webhook port to be configured #849

  • Encode “<” as u003c in response #837

  • Tensor wrapped Numpy is read-only #807

  • Kubeflow and Istio deployment - HTTP, gRPC endpoints not working #796

  • javaopts -Xmx throws error #784

  • Override webhook bootstrap default for non-default names #715

  • eliminate cluster wide Secret access RBAC #714

  • Services deployed outside of the helm charts do not get deleted #682

  • TLS handshake errors in manager logs #680

Merged pull requests:

v0.4.1 (2019-09-19)

Full Changelog

Implemented enhancements:

  • Investigate Edge/IOT deployments #414

  • NVIDIA Rapids Integration #412

  • Investigate Apache Arrow and Plasma for fast IPC #206

  • Explore creation of a “batching” transformer #107

Fixed bugs:

  • Unresolved variable in python utils #834

  • Seldon not creating services for NVIDIA TRT Deployment #826

  • Autoscaling not working #824

  • seldon-core-analytics grafana shows metrics of only of first canary model #618

  • DNS error in API Gateway for old deployments gRPC #262

Closed issues:

  • only install knative for request logging if not already present #869

  • Seldon Core deployment created but not visible #862

  • Default values of metrics missing in /feedback API for Python wrapper #861

  • Missing packages on requirements.txt #858

  • Seldon GPU Image with seldon-core 0.4.0 support not available #850

  • How do I override the default port for the flask application that seldon runs ? #848

  • Add documentation for our certified openshift operator #845

  • consume a Secret in a volume in a Pod #844

  • ModelUri is None when parameters specified in the SeldonDeployment #838

  • Meta data field not modifiable by subsequent components #829

  • Cannot disable secure connection to minio storage backend #825

  • How to send data in request with meta field #822

  • Combiner endpoint does not create VirtualService or Endpoint in Istio #816

  • Missing MLFLOW values in crd.yaml for helm chart 0.4.0 #811

  • support ambassador_id to allow multi-tenancy #810

  • High latency of REST locally #803

  • initContainer resources required for some clusters #776

  • Add versions to the wrapper requirements.txt and make sure they align with setup.py #767

  • “puid” when sent in request is not returned in response from python wrapper service. However in case of external api it is returned as is #765

  • Injecting S3 Credentials into tfserving Container #749

  • [Bug] REST request gets parsed to String if any of the element in Numpy array is String #745

  • Long running tasks not working with REST Clients (CURL, Postman, etc) #742

  • Ensure JSON format is not modified for ALL the REST APIs (only implemented for /predict) #739

  • Add the support to send image files directly using multipart/form-data instead of application/x-www-form-urlencoded #733

  • exception swallowing in seldon-core #725

  • versions for prepackaged model servers #724

  • Prometheus metrics "INVALID" is not a valid start token in kubernetes env #706

  • Change NotImplementedError in SeldonMethods for SeldonNotImplementedError #700

  • HttpRetryHandler interrupted exception leads to Microservice error: java.net.SocketTimeoutException: Read timed out (103) #698

  • combiner receives struct_pb2.ListValue instead of numpy list #686

  • explainers for SeldonDeployment graphs #685

  • Deployment with empty oauth key #683

  • Income Classifier Example with Kubeflow #676

  • installing sdep straight after operator seems not fully reliable #669

  • argo workflow not detecting seldon deployment is available #668

  • Set up node taints for GPU Pods/Nodes. #648

  • Update the Python GPU s2i files #647

  • volume mounting kubeflow example #637

  • docs in readthedocs for request logging and CICD #636

  • Seldon GRPC python wrapper doesn’t include input/output transformer servicers #630

  • NDArray with values being lists not supported - RESOLVED: Proto lists were not being deep-copied #600

  • Provide a NULL/None type in the CRD #554

  • Support microservice testing via curl with JSON content-type #552

  • Python client - SyntaxError #551

  • Add Prow doc with Public URL for dashboard #546

  • Investigate Request Logging #545

  • Support Kustomize Deployment #536

  • make engine probes configurable #525

  • connect timed out with cluster-manager and api server #523

  • put crd and controller into single helm chart #522

  • Add fastai deployment example #518

  • Deploy model but pod is evicted for many times before running #515

  • How to wrap models using tool chains other than s2i? #507

  • ci-cd demo uses old jenkins and kubectl fails on newer k8s #505

  • Prow Integration - investigate and setup PR presubmit jobs #497

  • Use of protobuffers for resource parsing cause inconsistencies #489

  • option to put prometheus metrics on separate port #487

  • parameterise the namespace in single namespace tests #478

  • use official helm charts #476

  • use recommended labels #475

  • Convert docstrings in serving_test_gen.py to Numpy format #472

  • issues on increasing grpc message size #467

  • Create a validating admission webhook for the CRD #459

  • Additional parameter types Seldon deployment proto #456

  • Consider hosting helm charts on stable repo, or at least a chart repository #443

  • Unable to get token #438

  • Update example Keras #432

  • Create protos for metadata returned by components #427

  • Revamp how the engine handles metadata #426

  • seldon-core-s2i-python3 is not “openshift ready” #422

  • Create docs for proto compilation to ease clients in non-python languages #421

  • KNative Integration #417

  • Julia Wrapper #413

  • Kaniko Integration #411

  • gRPC tensorproto, TypeError #410

  • Handle sticky model requests #403

  • Organize docs #394

  • Type annotations for Python code #392

  • Deploy python model without java router #314

  • Implement a hierarchy of tags for components with children. #310

  • FlatBuffers support #284

  • Custom fields in SeldonMessage #198

  • Update CRD to include version #175

  • Add OpenFaas as Deployment option for ServerLess ML deployments #151

  • Can Spartakus be run in a namespace #142

  • Investigate Google Container Builder for wrapping models #123

  • Investigate OpenShift Service Catalog and Brokers #114

  • Change python wrapper to use Tornado instead of Flask #108

  • Add docs to build components in development section #65

  • Clarify oauth key/secret provisioning #26

  • Create a NOTES.txt for the helm chart #15

Merged pull requests:

v0.4.0 (2019-08-15)

Full Changelog

Fixed bugs:

  • multiple containers in componentSpecs.spec.containers not supported? #718

  • Ambassador does not work in v0.3.2 #710

Closed issues:

  • Create kustomize for 0.3.2 release #794

  • Add redhat certified image to operatorhub #791

  • Build model servers with non-SNAPSHOT image #790

  • env secret for explainers #785

  • finding image names used in grafana dashboards for prepack model servers #783

  • seldon container engine sidecar java process should run with ‘-server’ setting #781

  • label selector problem when running xgboost and sklearn iris prepack examples together #777

  • Documentation links cannot be seen (just change css color) #773

  • [Bug]Invalid metadata label generated from deployment file #772

  • Add a basic MLFlow model server based on example #771

  • additionalProperties cannot be set to false error when installing helm chart #768

  • Cannot make GRPC calls through GRPC microservice wrapping. #762

  • make it clearer how and when to set engine resources #761

  • Update TFServing Proxy to use v0.5 of the python wrapper #760

  • intermittent 503 errors in end-to-end tests #758

  • Add global count Graphana graph per deployment #756

  • docs: python wrapping docker #754

  • Liveness probe kills seldon engine container when model predict function takes a long time to send a result #753

  • models from private buckets for prepackaged/standalone servers #748

  • prepackaged/standalone servers to download models from initContainers #747

  • Writing the predict function around pandas dataframes #746

  • Golang CRD api #734

  • Update Roadmap #723

  • CRD in kustomize is missing subresource.status #713

  • calling SeldonClient.predict inside a microservice model rasing “TypeError: Couldn’t build proto file into descriptor pool!” #711

  • Add SELDON_LOG_LEVEL parameter in Python Wrapper #708

  • All numbers get converted to float when sending JSON resulting in unexpected behaviour in Proxies #703

  • docs: ambassador version #697

  • docs: Python API reference #696

  • Back-off pulling image “data_downloader:0.1”: ImagePullBackOff #695

  • Deploying modified TensorRT example on GKE does not deploy anything #694

  • Support JSON format in TFServer Proxy REST API #693

  • Add build files for alibi explainer server #692

  • docs: Ambassador REST endpoint path #688

  • cannot run through the example. #677

  • Liveness probe kills Seldon engine container with long init waiting time (Python wrapper) #674

  • need to increase failure threshold for engine with new operator #673

  • Remove api-frontend #663

  • Ambassador Swagger definition does not include namespace #660

  • Remove Java Operator #654

  • Docs do not mention addition of jsonData outside of the release notes #652

  • Support Swagger or something like that #645

  • Changes required for Seldon Core Analytics Grafana to work behind proxy #622

  • Tensorflow GPU Example #619

  • HTTPS/HTTP switch for seldon client #572

  • How can I modify the logging format of REST API for StackDriver and BigQuery? #557

  • The output from ambassador converts integer types to float . #458

  • Question: Concurrency in Model REST server #453

  • Add model explainer example #425

  • Kubeflow Pipelines Integration #415

  • Add an WSGI app server to Python REST wrappers #383

Merged pull requests:

v0.3.1 (2019-06-28)

Full Changelog

Closed issues:

  • Embedded travis build status needs updating (in main readme) #656

  • Add doc link to prow #655

  • Remove Old Java Operator Docs #653

  • requests all failed during benchmarking with Locust #644

  • Seldon & Istio - broken link #643

  • Seldon-core-operator install fails #640

  • how to add custom meta fields for tracking #632

  • Using the cluster-manager underlying API as a library #626

  • User Exceptions raised in predict function of a wrapper class needs to be handled via flask exception handler #625

  • volume mounting - update examples #604

  • engine resources not configurable in new operator #598

  • status code in response missing. Seldon v 0.2.3 #519

  • deprecate ksonnet #477

Merged pull requests:

v0.3.0 (2019-06-05)

Full Changelog

Fixed bugs:

  • Python module syntax error for python 3.5 for variable type annotation #614

  • openvino_imagenet_ensemble example - prediction1 and prediction2 containers error #583

  • Docker image name and version messed up #562

  • SendonDeployment with name &gt; 31 characters #556

  • Not able be build python3.6 base image. #542

Closed issues:

  • Update master to 0.3.0 SNAPSHOT #612

  • sdep state doesn’t move to available #605

  • engine using 1 cpu #597

  • forbidden error installing ambassador #596

  • GPU support with SERVICE_TYPE Model #590

  • Update example notebooks for docs #586

  • kubeflow/example-seldon on local cluster #585

  • Unable to inject custom parameter in Python model #584

  • Tensorflow MNIST Model example on EKS #580

  • OOMKilled when starting an operator #579

  • Can we customize the outputs format of Model? #565

  • node exporter port conflict #563

  • How can i enable debug logging of seldon-engine container #560

  • Tensorflow Python 3.7 support and wrapper images #550

  • helm-charts upgrade fails on ambassador #543

  • Update Seldon Core Analytics Grafana #540

  • Defining the model serving class with full name doesn’t currently work #533

  • Update Seldon Operator to Go #529

  • Old Containers & Security Vulnerabilities #528

  • option to not set runAsUser for engine #527

  • Support istio ingress #511

  • Endpoint type is missing for AB-test #451

  • Alllow arbitrary JSON as a payload #434

  • Update Ambassador to 0.40.2 #402

  • Ambassador config for rolling updates #294

Merged pull requests:

v0.2.7 (2019-04-29)

Full Changelog

Implemented enhancements:

  • Type check predictive unit parameters in the Python wrapper #440

Fixed bugs:

  • Models pods duplications after corrupted deployment #470

  • Using a configMapRef inside of a seldon deployment manifest causes a NullPointerException in the SeldonDeploymentWatcher #450

  • cannot get working external api but internal api is ok #448

  • Status can become Available even with Exception in Operator #429

  • Fix status update for failed deployments #474 (cliveseldon)

Closed issues:

  • Install seldon in a single namespace with restricted tiller #514

  • Document about microservice’s input data #512

  • where is io.seldon.protos.DeploymentProtos package located? #508

  • seldon 0.2.3 - nfs volume in seldon graph failing in validation #504

  • SeldonDeployment keeps hanging #499

  • default ambassador chart to single namespace #495

  • use v1 ambassador api #491

  • Configure the way Prometheus exposed #484

  • documentation is in doc not docs #481

  • do a snapshot build and document if not documented #479

  • How can we specify nested python class in .s2i/environment? #465

  • Class names in latest python library is not backwards compatible #462

  • Sending an object dtype array as the request JSON for a Model API #461

  • NullPointer exception in API gateway when principal can’t be determined #454

  • Python Wrappers Version 2 #406

  • Write a Python wrapper for a GENERIC component #378

  • Create reference Python client #349

  • Python-wrapper: Use debug flag to provide useful information #309

  • Support autoscaler in SeldonDeployment #277

  • Update Ambassdor Helm or remove and use Ambassador’s helm chart #258

  • Prow Integration #154

  • CI/CD demo using GitOps framework #11

Merged pull requests:

v0.2.6 (2019-02-22)

Full Changelog

Fixed bugs:

  • Bug parsing boolean predictive unit params in Python wrappers #439

  • APIFE fails to connect to service due to name change #433

Closed issues:

  • If building a python image from a folder, which is also a git-folder build silently fails #452

  • Setting engineResources not enabling resource requests/limits to seldon-container-engine sidecar #398

  • Expose Jaeger agent port as environment variable on deployment manifest #396

  • Ksonnets for Seldon Analytics #391

  • sklearn iris returns value error #389

  • SOAP API #387

  • unable to find proto file which defines grpc #384

  • tensorflow-gpu #380

  • onnx_resnet50.ipynb : “Unknown operation: Gather” #379

  • Passing arguments to the model object #377

  • Model pod enters in CrashLoopBackOff. How to debug? #376

  • Global metrics show N/A in Seldon Analytics Grafana #371

  • Mistyped check causing NULL Pointer Exceptions with getNamespace #367

  • Json payload size increases when I use json.dumps #365

  • Need an updated tutorial for seldon serving on GKE #361

  • Hi,we need Golang Deploy Seldon Wrapper Container #356

  • Update docs and examples to use the new Python package #347

  • Potential problem in EpsilonGreedy.py? #336

  • Deploying seldon-core to Kubernetes 1.8.6 fails with no matches for kind "Deployment" in version "apps/v1" #333

  • S2i build image with private pip repository #330

  • Wrapping components outside of the tree #324

  • Seems to be a bad fit for a multi-tenant cluster. #308

  • Update Grafana / Prometheus image #303

  • Function to pass additional meta info for predict\(\) #297

  • Update base java image #289

  • Update ksonnet to reflect latest helm templates #282

  • NullPointerException in seldon-cluster manager logs #268

  • requestPath picking up old model on rolling update #267

  • Seldon deployment success/failure condition #255

  • Reconcile the differences between seldon-core and kubeflow core.libsonnet to improve maintenance #237

  • Make the “apiVersion” in the Helm templates consistent #236

  • Create initial docs for Transformers #229

  • Create initial docs for Routers #228

  • deploy docker image is ok ,but deploy k8s pod always failed #212

  • Format of the data sent as a request to the seldon REST api? #193

  • There is no setting that allows increasing the limits of GRPC Server #183

  • Docker image build error with sklearn_iris_docker example #164

  • Add support for spring-boot-starter-webflux #152

  • gRPC query waits indefinitely while execution giving no output #149

  • scikit-learn support for predict method not only predict_proba #145

  • Wrapper command on windows PS #134

  • Error 401 while requesting prediction outputs from seldon server #122

  • How to Deploy our custom models on seldon-core #104

  • Create docs for available plugins #100

  • Custom model endpoints #96

  • Docker image missing for Iris classification #91

  • Add options to populate meta data in wrappers foreach API request #86

  • Add InputOutputTransformer predictive unit #85

  • Add Explainer as transformer component #84

  • Create wrapper for PyTorch models #82

  • Graph with epsilon greedy router sometimes fails on first request #80

  • Create integration testing script #73

  • Allow engine resource requests for engine to be configurable in proto definition for CRD #70

  • Create Concept Drift Alert Plugin #56

  • Create Outlier Detection Plugin #55

  • Create Multi-Armed Bandit Router Plugin(s) #54

  • Update docs for sklearn_iris_docker #52

  • Response should contain indication of which predictor was used #50

  • Add git hooks for validation of notebooks before commit #10

  • Update docs and examples to illustrate complex runtime graphs #1

Merged pull requests:

v0.2.5 (2018-12-16)

Full Changelog

Closed issues:

  • initialDelaySeconds: 10 sec is not enough for some models #323

  • Bug: custom metrics for both children and parent components #322

  • Ambassador seldon deployment not registered #318

  • Allow user to disable Redis in seldon-core helm chart #304

  • grpc tensor convert not valid for python 2 #301

  • Ambassador #298

  • Create a python wrapper for COMBINER components #296

  • packaging for python microservice wrapper #293

  • Update to latest Spartakus image #291

  • Docker image “seldonio/seldon-core-s2i-python3” uses old Python 3.6 #288

  • Seldon cluster-manager k8s cluster wide operations #269

  • S2I hangs in example models when using minikube docker-env #253

  • Accessing custom metrics in our Python model #245

  • strData & binData not accepted by Python model microservice #225

  • Utilize latest /status endpoint for Custom Resources in k8s 1.11 #176

  • Investigate Nvidia’s TensorRT #121

  • Review status field for CRD #83

  • gitops demo #67

  • Update seldon-core/examples docs after move #35

  • Create wrapper for Spark standalone runtime models #24

Merged pull requests:

v0.2.4 (2018-11-07)

Full Changelog

Closed issues:

  • Specification of a Service Account #286

  • curl not found error #283

  • Allow ambassador from other namespace to access SeldonDeployment #279

  • Fix Github security vulnerabilities in dependencies #259

  • Feedback API not called when using Models #251

  • Allow JAVA OPTS for engine to be specified #249

  • ndarray greater than 15280 bytes #248

  • Prediction API get model version #244

  • SeldonDeployment creation strips out an empty “children” list field in the manifest #242

  • Mahalanobis Outlier Detector fails when batch is of size 1 #240

  • Seldon Core Operator defaulting causes issues with helm and ArgoCD #233

  • TensorFlow Serving as the Model microservice #226

  • Nodejs wrapper for javascript models #216

  • Environmental variable error #215

  • How do I increase timeout of sidecar seldon container? #196

  • update release script for pyhton3 #160

  • Ability to customize Ambassador configuration #120

  • Script to convert proto files and generate OpenAPI schema #9

  • OpenAPI spec for external and internal prediction APIs #6

Merged pull requests:

v0.2.3 (2018-09-17)

Full Changelog

Closed issues:

  • R wrapper s2i environment documentation missing Model file extension #219

  • Provide example using ONNX via Intel nGraph for inference #214

  • how to explore grafana dashboard for seldon-core in web ui #209

  • APPLICATION FAILED TO START - Example python notebook (fx-market-predictor) #208

  • dev guide doc: develop/test changes locally? #202

  • Service orchestrator updated when surrounding deployment changed #199

  • UnknownHostException: seldon-deployment #194

  • grafana dashboard #192

  • Add image versions to all wrapper images #136

  • Status is not created in Custom Resource on initial create or update #74

Merged pull requests:

v0.2.2 (2018-07-18)

Full Changelog

Merged pull requests:

v0.2.1 (2018-07-09)

Full Changelog

Closed issues:

  • “/s2i/bin/assemble: line 59: 10 Killed” when using S2I to build PyTorch container #180

  • Getting timeout error using S2I to package PyTorch model #179

  • Make Operator Create CRD on StartUp #174

Merged pull requests:

v0.2.0 (2018-06-29)

Full Changelog

Closed issues:

  • PREDICTIVE_UNIT_PARAMETERS: not able to set them up correctly #170

  • Add docs page for Helm #71

Merged pull requests:

v0.1.8 (2018-06-27)

Full Changelog

Closed issues:

  • Prediction analytics dashboard not capturing prediction API calls #168

  • Ambassador + Minikube doc needs to be updated per RBAC #165

  • configmap type volume gets mounted as EmptyDir #162

  • Java Wrapper H2OUtils doesn’t check type in NDArray proto message conversion #158

  • Release Java wrappers library 0.1.1 #157

  • Automate update of ksonnet versions in release process #132

Merged pull requests:

v0.1.7 (2018-06-04)

Full Changelog

Closed issues:

  • Quickstart problem #153

  • NameError: global name ‘ListValue’ is not defined #148

  • bad credentials error with get_token function #144

  • Make CRD Namespaced scoped #141

  • Create wrappers for Java based models #137

  • Update ksonnet prototypes for latest image version #130

  • Create demo notebook for Azure #129

  • Grafana Dashboard #109

  • Multiple helm seldon-core installs on separate namespaces fails #106

Merged pull requests:

v0.1.6 (2018-03-29)

Full Changelog

Closed issues:

  • Support RBAC by default #126

  • Engine requires images to have versions #117

  • hostPath type volume gets mounted as emptyDir #116

  • Investigate OpenShift source-to-image for wrapping models #113

  • Add docs for analytics persistence #112

  • Issue in deployments of multiple models #103

  • Missing dependencies in notebooks/kubectl_demo_minikube.ipynb #101

  • Add usage metrics collector #99

  • Running test model on seldon core #90

  • Deploying seldon models to multiple namespaces #89

  • Generate load tests analytics #58

  • Create wrapper for R models #23

Merged pull requests:

v0.1.5 (2018-02-19)

Full Changelog

Closed issues:

  • Make CRD namespaced #95

  • Allow Helm deployment without API Front end #92

  • Support deployment of a Python 3 model #88

  • Create a Slack channel for project #43

Merged pull requests:

v0.1.4 (2018-02-05)

Full Changelog

Closed issues:

  • Create KSonnet prototypes for core #76

  • Add automatically generated README to wrapped models #57

Merged pull requests:

v0.1.3 (2018-01-26)

Full Changelog

Fixed bugs:

  • Insufficient cpu error when creating complex graphs #47

Closed issues:

  • Split Prometheus monitoring from seldon-core Helm Chart #69

  • Docs for how to deploy and CI/CD options #30

Merged pull requests:

v0.1.2 (2018-01-23)

Full Changelog

Closed issues:

  • Remove the cluster manager functionality that checks for “type” presence in predictive units graph #59

  • Change builds to use seldonio/core-builder:0.2 #48

  • Cluster manager stuck in an error loop after failed deployment #45

  • Quickstart in minikube has old resource spec for deployment #40

  • Bring seldon-core-examples into main project #33

  • Update travis integration for all builds #25

  • Update model wrapping docs and docker wrapping code #17

  • Links are broken on the following doc page #16

Merged pull requests:

v0.1.1 (2018-01-10)

Full Changelog

Closed issues:

  • GRPC microservices fail kubernetes health and readiness checks #13

  • Update diagrams in docs to reflect latest version of the protos #7

  • Average Combiner broken in container #5

  • external gRPC client not updating when deployment removed and recreated #4

  • Cluster Manager failing to authenticate on GKE cluster #2

Merged pull requests:

v0.1.0 (2018-01-03)

Full Changelog