Brainstorming: What do you want in ddt v2.0.0?

[pradyunsg]

I noticed that @txels had expressed the desire to ditch backwards compatibility and give ddt another shot to get a more powerful platform on the other side. Hence, with the intent of sparking discussion, I write this.

[pradyunsg]

I would like to be able to specify the name of a generated function, cleanly. Like test_1_is_less_than_10 instead of test_is_less_than_10_1_1.

I also noticed that Cartesian products of parameters is another popular request.

[htv2012]

At work, I am using ddt every day and one of the features I am looking for is the ability to have per-ddt set-up/tear-down. In keeping up with the naming pattern, I propose to name them setUpDdt and tearDownDdt. I already made a fork, tried out the idea myself and it seems to work. I need to work on the testing and documentation part.

[txels]

Hi all, thanks for using the library, your contributions, and especially for not bitching about me not being very responsive :)

I have been giving the future of ddt some thought over the past weeks.

To give some background about how ddt came to be: I was looking for something that gave to unit tests a similar power as to what one could do with nosetests generators (which only worked for nose function-based tests): the ability to run a test case multiple times with different data. At that time I was beginning to learn me some more advanced Python (I had only been using the language for a few months) and was exploring the various mechanisms for metaprogramming available in the language. I was fascinated that one could add methods to a class at runtime, and I wrote this then tiny library that did that.

Over time I've been taking some people's contributions, some of which grew the feature set into areas I hadn't even considered (like reading data from files) - but in honesty, that tiny initial feature has been enough for me and is about the only one I've actually personally used from the library.

This has unfortunately meant that I've not always been paying as much attention as I should have to backwards compatibility of certain features (which made some people not particularly happy) or simplifying extensibility in general.

Back from that digression, I've often thought that I wanted ddt to go back to its simpler origins, but still be easily applicable to other people's more complex use cases. One of the design decisions I've come to regret the most about ddt has been using *args as the mechanism to pass test data to the test methods, because that limits the potential for the API to grow quite a bit.

My current thinking of what the new ddt should look like is:

• keep the class decorator ddt
• a single method decorator that takes test data as an iterable (e.g. similar to the one proposed in Add @idata(iterable) decorator-function. #40), and options as keyword arguments. A well defined mechanism would tell ddt how to generate suffixes for the test cases from the test data.
• get rid of all the other decorators for unpacking, reading from file etc.
  • the unpacking options would become keyword arguments to the main decorator (e.g. unpack, unpack_dict or similar...)
  • the methods to read from file etc would be just methods that read from a file and return some iterable over the data

E.g. assuming we call the new decorator idata, it would look like:

@idata(test_data, **options)
def test_something(self, value):

E.g. in a practical use:

@idata(from_file('something.yml'), unpack=TUPLE)
def test_something(self, x, y, z):

@idata(some_dict, unpack=DICT)
def test_something_else(self, x, y, result):

Requested features like cartesian products (or any test data generation algorithm) could easily be added as just plain functions with whatever arguments that return an iterable. In fact cartesian = itertools.product - that is already available in the standard library and we don't need to include one.

@idata(itertools.product(dataset_a, dataset_b), unpack=TUPLE)
def test_something_else(self, a_value, b_value):

You could then easily add features similar to those of hypothesis to generate special data for boundary testing.

I'm of course open to any suggestions and constructive criticism.

[pradyunsg]

I would actually also like to add the ability to change the naming algorithm as an argument to the global class decorator. If ddt adds a fake module (ddt.helpers), it'll be the perfect place to put all the support functions, like file and all...

Could you look into whether you can support unittests' setup and teardown functions? It shouldn't be too hard.

I'm +1 to all the changes you mention.

[pradyunsg]

Oh, and if someone passes a literal-like value (list, tuple, dict) to idata suggest that they use data instead.

[pradyunsg]

I know that ddt currently just adds the tests with data to the class... Would it be possible to get the data without loading it into memory at class creation time? As in do the setup of the methods in ddt decorator but really load the data at runtime?

[txels]

@htv2012 I'm not sure what is your use case for setUpDdt that isn't covered with regular setUp or setUpClass methods. Can you elaborate? When and how would you use that?

[txels]

@pradyunsg

Would it be possible to get the data without loading it into memory at class creation time? As in do the setup of the methods in ddt decorator but really load the data at runtime?

I don't see how that could be possible without having to iterate through data more than once. It would complicate the implementation for what IMHO is a narrow use case (large data samples). To someone wanting to test large amounts of data or very large data samples in a memory-efficient way, I'd recommend to use a different approach than ddt.

I would actually also like to add the ability to change the naming algorithm as an argument to the global class decorator.

This sounds good. Truth be told, I was considering something in that line (as part of **options, similar to the unpacking mechanisms), but I don't have a concrete approach to implementation in mind yet. Making it possible to set global options in the ddt decorator that act as defaults to idata is something I was also considering, so it's good to see someone else thinking in that direction.

[htv2012]

@pradyunsg

What if I want to pass the test data to setUp so it can do some prep work based on the data? Currently, we cannot do that as setUp does not accept any parameter. For example, if my test class needs to perform the following prep works: Given a spreadsheet file, the name of the tab, and the coordinate of the cell (these are part of my test data), open the spread sheet, go to the specified tab, then go to the specified cell, read the value off that cel. With the normal setUp, I cannot pass in those test data.

[pradyunsg]

@htv2012 I don't understand why you would want to load the data in setUp. The way I see it, that should really be part of the test itself, passed as a parameter by ddt.

What I was thinking is that if it would be possible to have some sort of switch in ddt that'll make it nicer to use for larger data-sets, it would be nice. I don't have a use-case at hand right now.

[pradyunsg]

Bump.

[pradyunsg]

Cleaning up my personal issues list - closing this.

[ntindle]

@htv2012 Do you still have that setup and teardown code around? I know its been several years but it seems very useful

[htv2012]

Hello Nicholas, I regret not keeping that code. My current job does Python, but not unittest, and definitely note ddt. I would like to take this opportunity to say that when I used DDT a few years back, I really appreciate the library. Thank you and keep up the good work. Hai

…

On Mon, Jul 22, 2019 at 12:06 PM Nicholas Tindle ***@***.***> wrote: @htv2012 <https://github.com/htv2012> Do you still have that setup and teardown code around? I know its been several years but it seems very useful — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#44?email_source=notifications&email_token=AAXA3XV5V2MOOFCUHJIU623QAYAL7A5CNFSM4CFOTFVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2Q3IPA#issuecomment-513913916>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXA3XQKDWZFAB2NZACYYGLQAYAL7ANCNFSM4CFOTFVA> .