History and Future

Versioning

Rpclib respects Semantic Versioning rules outlined in http://semver.org.

This means you can do better than just adding 'rpclib' to your list of dependencies. Assuming the current version of rpclib is 2.4.8, you can use the following as dependency string:

  • rpclib if you feel adventurous or are willing to work on rpclib itself.
  • rpclib<3 if you only want backwards-compatible bug fixes and new features.
  • rpclib<2.5 if you only want backwards-compatible bug fixes.
  • rpclib=2.4.8 if you rather like that version.

We have a policy of pushing to pypi as soon as possible, so be sure to at least use the second option to prevent things from breaking unexpectedly.

Rpclib project uses -alpha -beta and -rc labels to denote unfinished code. We don’t prefer using separate integers for experimental labels. So for example, instead of having 2.0.0-alpha47, we’ll have 2.2.5-alpha.

  • -alpha means unstable code. You may not recognize the project next time you look at it.
  • -beta means stable(ish) api, unstable behavior, there are bugs everywhere! Don’t be upset if some quirks that you rely on disappear.
  • -rc means it’s been working in production sans issues for some time on beta-testers’ sites, but we’d still like it to have tested by a few more people.

These labels apply to the project as a whole. Thus, we won’t tag the whole project as beta because some new feature is not yet well-tested, but we will clearly denote experimental code in its documentation.

Roadmap and Criticism

This is an attempt to make a free-for-all area to display opinions about the feature direction of rpclib. Doing it in a text file in the source repository may not be the best approach for the job, but it should be enough to at least spark a discussion around this topic.

So the following is missing in Rpclib:

Processing Pipeline

We think rpclib package has one last missing element whose addition can result in touching most of the codebase: A proper lazily-evaluated pipeline for request processing.

Currently, every artifact of the rpc processing pipeline remain in memory for the entire life time of the context object. This also results in having the whole message in memory while processing. While this is not a problem for small messages, which is rpclib’s main target, it limits rpclib capabilities.

Serializer Support

See the High-Level Introduction to Rpclib section for a small introductory paragraph about serializers.

Currently, serializers are not distinguished in the rpclib source code. Making them pluggable would:

  1. Make rpclib more flexible
  2. Make it easy to share code between protocols.

An initial attempt to make them pluggable would result in the lxml dependency for Soap being relaxed, which would make it possible to deploy rpclib in pure-python environments. However, this is comparatively easy to do, given the fact that the ElementTree api is a well-known de-facto standard in the Python world.

Adding other serializers like json to the mix would certainly be a nice exercise in oo interface design, but this may be a solution in search of a problem. Would anybody be interested using Soap over Json instead of Xml? Probably not :)

It would, however, help newer serialization formats by reusing code from their more mature cousins. E.g. Soap already has a security layer defined. If the serializer is abstracted away, it could be easier to port security code from Soap to JsonRpc.

Miscellanous

The following would definitely be nice to have, but are just modules that should not cause a change in unrelated areas of rpclib. Those would increment the minor version number of the Rpclib version once implemented.

  • Support for polymorphism in XML Schema.
  • Support for the JsonObject (à la XmlObject) and JsonRpc protocols.
  • Support for the JsonSchema interface document standard.
  • Support for the Thrift binary protocol.
  • Support for the Thrift IDL – The Thrift Interface Definition Language.
  • Support for the XmlRpc standard.
  • Support for EXI – The Efficient Xml Interchange as a serializer.
  • SMTP as server transport.
  • SMTP as client transport.
  • Improve HttpRpc to be Rest compliant. Probably by dumping HttpRpc as it is and rewriting it as a wrapper to Werkzeug or a similar WSGI library.
  • Implement converting csv output to pdf.
  • Implement DNS as transport
  • Support security extensions to Soap (maybe using PyXMLSec ?)
  • Support addressing (routing) extensions to Soap
  • Add WSDL Parsing support to Soap client
  • Reflect transport and protocol pairs other than Soap/Http to the Wsdl.

Comparison with other rpc frameworks

RPC is a very popular subject. There’s a plethora of active and inactive projects that satisfy a wide range of requirements. Here are the main sources of information:

Ladon

The Ladon project has almost the same goals and same approach to the rpc-related issues as rpclib.

Discussion thread: https://answers.launchpad.net/ladon/+question/171664

  • Supports JsonWSP protocol, which rpclib does not support.

    The main motive for designing JSON-WSP was the need for a JSON-based RPC protocol with a service description specification with built-in service / method documentation.

  • Supports both Python 2 and Python 3.

  • Auto-generates human-readable API documentation. (example: http://ladonize.org/python-demos/AlbumService) In Rpclib, you need to do with the ugliness of a raw wsdl document.

  • Does not support ZeroMQ.

  • Uses standard python tools for xml parsing which is good for pure-python deployments. Rpclib uses lxml, due to its excellent namespace support and speed. So Rpclib-based solutions are easier to develop and faster to work with but more difficult to deploy.

  • Does not do input validation for SOAP.

  • Does not support events.

  • Does not support HttpRpc.

  • Does not have a Soap client.

    In fact, Ladon is pure server-side software - the whole idea of supporting a standard protocol like SOAP is that clients are already out there.

  • Rpclib uses own classes for denoting types, whereas ladon uses Python callables. This lets ladon api to be simpler, but gives the rpclib api the power to have declarative restrictions on input types.

  • Does not test against ws-i.org deliverables for testing soap compatibility.

  • Does not support parsing and/or modifying protocol & transport headers.

WSME

“”” Web Service Made Easy (WSME) is a very easy way to implement webservices in your python web application. It is originally a rewrite of TGWebServices with focus on extensibility, framework-independance and better type handling. “”“

  • Supports TurboGears
  • Supports REST+Json, REST+Xml, (a subset of) SOAP and ExtDirect.
  • Supports type validation.
  • No client support.
  • Does not test against ws-i.org deliverables for testing soap compatibility.
  • Only supports http as transport.
  • Uses genshi for Xml support.

RPyC

This is preliminary. Please correct these points if you spot any error.

  • Uses own protocol
  • Does not do validation.
  • Python-specific.
  • Fast.
  • Not designed for public servers. ??

rfoo

This is preliminary. Please correct these points if you spot any error.

  • Uses own protocol
  • Does not do validation.
  • Python-specific.
  • Fast.
  • Not designed for public servers. ??

Suds

  • Soap 1.1 / Wsdl 1.1 Client only.
  • Excellent wsdl parser, very easy to use.
  • Recommended way to interface with rpclib services.
  • Uses own pure-python xml implementation, so it’s roughly 10 times slower than rpclib.
  • Only depends on pure-python solutions, so much easier to deploy.

ZSI

  • Unmaintained, although still works with recent Python versions
  • Contains SOAPpy, which is not the same as SOAPy (notice the extra P)
  • Supports attachments
  • Requires code generation (wsdl2py) for complex data structures
  • Almost complete lack of user-friendliness
  • Lack of WSDL generator

SOAPy

  • Really simple (only two files, both less than 500 lines of code)
  • Client only
  • Requires PyXML, thus unusable with recent Python versions

rsl

  • Client only.
  • Unmaintained.

PyRo

  • ???

Changelog

rpclib-2.5.2-beta

rpclib-2.5.1-beta

rpclib-2.5.0-beta

  • Implemented fanout support for transports and protocols that can handle that.
  • Implemented a helper module that generates a Soap/Wsdl 1.1 application in rpclib.util.simple
  • Some work towards supporting Python3 using 2to3. See issue #113.
  • ctx.descriptor.reset_function implemented. It’s now safe to fiddle with that value in event handlers.
  • Incorporated a cleaned-up version of the Django wrapper: https://gist.github.com/1316025
  • Fix most of the tests that fail due to api changes.
  • Fix Http soap client.
  • Full change log: https://github.com/arskom/rpclib/pull/115

rpclib-2.4.7-beta

  • Made color in logs optional
  • Fixed ByteArray serializer

rpclib-2.4.5-beta

rpclib-2.4.3-beta

  • Many issues with ‘soft’ validation was fixed.
  • MethodDescriptor.udp added. Short for “User-Defined Properties”, you can use it to store arbitrary metadata about the decorated method.
  • Fix HttpRpc response serialization.
  • Documentation updates.

rpclib-2.4.1-beta

  • Fixed import errors in Python<=2.5.
  • A problem with rpclib’s String and unicode objects was fixed.

rpclib-2.4.0-beta

  • Fixed Fault publishing in Wsdl.
  • Implemented ‘soft’ validation.
  • Documentation improvements. It’s mostly ready!
  • A bug with min/max_occurs logic was fixed. This causes rpclib not to send null values for elements with min_occurs=0 (the default value).
  • Native value for rpclib.model.primitive.String was changed to unicode. To exchange raw data, you should use rpclib.model.binary.ByteArray.
  • Full change log: https://github.com/arskom/rpclib/pull/90

rpclib-2.3.3-beta

  • Added MAX_CONTENT_LENGTH = 2 * 1024 * 1024 and BLOCK_LENGTH = 8 * 1024 constants to rpclib.server.wsgi module.
  • rpclib.model.binary.Attachment is deprecated, and is replaced by ByteArray. The native format of ByteArray is an iterable of strings.
  • Exception handling was formalized. HTTP return codes can be set by exception classes from rpclib.error or custom exceptions.
  • Full change log: https://github.com/arskom/rpclib/pull/88

rpclib-2.3.2-beta

  • Limited support for sqlalchemy.orm.relationship (no string arguments)
  • Added missing event firings.
  • Documented event api and fundamental data structures (rpclib._base)
  • Full change log: https://github.com/arskom/rpclib/pull/87

rpclib-2.3.1-beta

rpclib-2.3.0-beta

  • Documentation improvements.
  • rpclib.protocol.xml.XmlObject is now working as out_protocol.
  • Many fixes.

rpclib-2.2.3-beta

  • Documentation improvements.
  • rpclib.client.http.Client -> rpclib.client.http.HttpClient
  • rpclib.client.zeromq.Client -> rpclib.client.zeromq.ZeroMQClient
  • rpclib.server.zeromq.Server -> rpclib.server.zeromq.ZeroMQServer
  • rpclib.model.table.TableSerializer -> rpclib.model.table.TableModel

rpclib-2.2.2-beta

  • Fixed call to rpclib.application.Application.call_wrapper
  • Fixed HttpRpc server transport instantiation.
  • Documentation improvements.

rpclib-2.2.1-beta

  • rpclib.application.Application.call_wrapper introduced
  • Documentation improvements.

rpclib-2.2.0-beta

  • The serialization / deserialization logic was redesigned. Now most of the serialization-related logic is under the responsibility of the ProtocolBase children.
  • Interface generation logic was redesigned. The WSDL logic is separated to XmlSchema and Wsdl11 classes. ‘add_to_schema’ calls were renamed to just ‘add’ and were moved inside rpclib.interface.xml_schema package.
  • Interface and Protocol assignment of an rpclib application is now more explicit. Both are also configurable during instantion. This doesn’t mean there’s much to configure :)
  • WS-I Conformance is back!. See https://github.com/arskom/rpclib/blob/master/src/rpclib/test/interop/wsi-report-rpclib.xml for the latest conformance report.
  • Numeric types now support range restrictions. e.g. Integer(ge=0) will only accept positive integers.
  • Any -> AnyXml, AnyAsDict -> AnyDict. AnyAsDict is not the child of the AnyXml anymore.
  • rpclib.model.exception -> rpclib.model.fault.

rpclib-2.1.0-alpha

  • The method dispatch logic was rewritten: It’s now possible for the protocols to override how method request strings are matched to methods definitions.
  • Unsigned integer primitives were added.
  • ZeroMQ client was fixed.
  • Header confusion in native http soap client was fixed.
  • Grouped transport-specific context information under ctx.transport attribute.
  • Added a self reference mechanism.

rpclib-2.0.10-alpha

  • The inclusion of base xml schemas were made optional.
  • WSDL: Fix out header being the same as in header.
  • Added type checking to outgoing Integer types. it’s not handled as nicely as it should be.
  • Fixed the case where changing the _in_message tag name of the method prevented it from being called.
  • SOAP/WSDL: Added support for multiple {in,out}_header objects.
  • Correct some bugs with the XMLAttribute model

rpclib-2.0.9-alpha

  • Added inheritance support to rpclib.model.table.TableSerializer.

rpclib-2.0.8-alpha

  • The NullServer now also returns context with the return object to have it survive past user-defined method return.

rpclib-2.0.7-alpha

  • More tests are migrated to the new api.
  • Function identifier strings are no more created directly from the function object itself. Function’s key in the class definition is used as default instead.
  • Base xml schemas are no longer imported.

rpclib-2.0.6-alpha

  • Added rpclib.server.null.NullServer, which is a server class with a client interface that attempts to do no (de)serialization at all. It’s intended to be used in tests.

rpclib-2.0.5-alpha

  • Add late mapping support to sqlalchemy table serializer.

rpclib-2.0.4-alpha

  • Add preliminary support for a sqlalchemy-0.7-compatible serializer.

rpclib-2.0.3-alpha

  • Migrate the HttpRpc serializer to the new internal api.

rpclib-2.0.2-alpha

  • SimpleType -> SimpleModel
  • Small bugfixes.

rpclib-2.0.1-alpha

  • EventManager now uses ordered sets instead of normal sets to store event handlers.
  • Implemented sort_wsdl, a small hack to sort wsdl output in order to ease debugging.

rpclib-2.0.0-alpha

  • Implemented EventManager and replaced hook calls with events.

  • The rpc decorator now produces static methods. The methods still get an implicit first argument that holds the service contexts. It’s an instance of the MethodContext class, and not the ServiceBase (formerly DefinitionBase) class.

  • The new srpc decorator doesn’t force the methods to have an implicit first argument.

  • Fixed fault namespace resolution.

  • Moved xml constants to rpclib.const.xml_ns

  • The following changes to soaplib were ported to rpclib’s SOAP/WSDL parts:
    • duration object is now compatible with Python’s native timedelta.
    • WSDL: Support for multiple <service> tags in the wsdl (one for each class in the application)
    • WSDL: Support for multiple <portType> tags and multiple ports.
    • WSDL: Support for enumerating exceptions a method can throw was added.
    • SOAP: Exceptions got some love to be more standards-compliant.
    • SOAP: Xml attribute support
  • Moved all modules with packagename.base to packagename._base.

  • Renamed classes to have module name as a prefix:
    • rpclib.client._base.Base -> rpclib.client._base.ClientBase
    • rpclib.model._base.Base -> rpclib.model._base.ModelBase
    • rpclib.protocol._base.Base -> rpclib.protocol._base.ProtocolBase
    • rpclib.server._base.Base -> rpclib.server._base.ServerBase
    • rpclic.service.DefinitionBase -> rpclib.service.ServiceBase
    • rpclib.server.wsgi.Application -> rpclib.server.wsgi.WsgiApplication
  • Moved some classes and modules around:
    • rpclib.model.clazz -> rpclib.model.complex
    • rpclib.model.complex.ClassSerializer -> rpclib.model.complex.ComplexModel
    • rpclib.Application -> rpclib.application.Application
    • rpclib.service.rpc, srpc -> rpclib.decorator.rpc, srpc

soaplib-3.x -> rpclib-1.1.1-alpha

  • Soaplib is now also protocol agnostic. As it now supports protocols other than soap (like Rest-minus-the-verbs HttpRpc), it’s renamed to rpclib. This also means soaplib can now support multiple versions of soap and wsdl standards.
  • Mention of xml and soap removed from public api where it’s not directly related to soap or xml. (e.g. a hook rename: on_method_exception_xml -> on_method_exception_doc)
  • Protocol serializers now return iterables instead of complete messages. This is a first step towards eliminating the need to have the whole message in memory during processing.

soaplib-2.x

  • This release transformed soaplib from a soap server that exclusively supported http to a soap serialization/deserialization library that is architecture and transport agnostic.
  • Hard dependency on WSGI removed.
  • Sphinx docs with working examples: http://arskom.github.com/rpclib/
  • Serializers renamed to Models.
  • Standalone xsd generation for ClassSerializer objects has been added. This allows soaplib to be used to define generic XML schemas, without SOAP artifacts.
  • Annotation Tags for primitive Models has been added.
  • The soaplib client has been re-written after having been dropped from recent releases. It follows the suds API but is based on lxml for better performance. WARNING: the soaplib client is not well-tested and future support is tentative and dependent on community response.
  • 0mq support added.
  • Twisted supported via WSGI wrappers.
  • Increased test coverage for soaplib and supported servers

soaplib-1.0

  • Standards-compliant Soap Faults
  • Allow multiple return values and return types

soaplib-0.9.4

  • pritimitive.Array -> clazz.Array
  • Support for SimpleType restrictions (pattern, length, etc.)

soaplib-0.9.3

  • Soap header support
  • Tried the WS-I Test first time. Many bug fixes.

soaplib-0.9.2

  • Support for inheritance.

soaplib-0.9.1

  • Support for publishing multiple service classes.

soaplib-0.9

  • Soap server logic almost completely rewritten.
  • Soap client removed in favor of suds.
  • Object definition api no longer needs a class types: under class definition.
  • XML Schema validation is supported.
  • Support for publishing multiple namespaces. (multiple <schema> tags in the wsdl)
  • Support for enumerations.
  • Application and Service Definition are separated. Application is instantiated on server start, and Service Definition is instantiated for each new request.
  • @soapmethod -> @rpc

soaplib-0.8.1

  • Switched to lxml for proper xml namespace support.

soaplib-0.8.0

  • First public stable release.