Skip to content

[MetaSchedule][BugFix] Fix skipped tests#10885

Merged
kparzysz-quic merged 1 commit into
apache:mainfrom
junrushao:feature/2022-04-02/bugfix-tune-config-api-change
Apr 2, 2022
Merged

[MetaSchedule][BugFix] Fix skipped tests#10885
kparzysz-quic merged 1 commit into
apache:mainfrom
junrushao:feature/2022-04-02/bugfix-tune-config-api-change

Conversation

@junrushao

@junrushao junrushao commented Apr 2, 2022

Copy link
Copy Markdown
Member

After #10366 is merged, all the search configurations (EvolutionarySearchConfig, ReplayFuncConfig, ReplayTraceConfig) are updated with a new field max_trials_global to be consistent with AutoScheduler's gradient-based task scheduler, which allows users to control the total number of trials of a model, while the scheduler distribute them unevenly according to task importance. This PR fixes the test cases that are not updated with this change, which are left undetected previously because they are not run in the CI.

For example, when tuning BERT-base, we could set max_trials_global to 20000, and the task scheduler will distribute trials to each task by preferring tasks with more potential performance growth:

 ID |                                                              Name |      FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  0 |                                                        fused_take |         1 |      1 |         0.0005 |       2.1504 |                2.1504 |      5 |          Y
  1 |                                      fused_nn_dense_add_fast_tanh |   1204224 |      1 |       139.7438 |       8.6174 |                8.6174 |    192 |          Y
  2 |                       fused_reshape_add_reshape_transpose_reshape |     49152 |     12 |        19.5901 |       2.5090 |               30.1083 |      5 |          Y
  3 |                                                    fused_variance |    147520 |     25 |        68.8360 |       2.1431 |               53.5766 |     45 |          Y
  4 |                                                        fused_mean |     49216 |     25 |        22.8079 |       2.1578 |               53.9462 |     45 |          Y
  5 |                                               fused_cast_take_add |     49152 |      1 |        21.4777 |       2.2885 |                2.2885 |      5 |          Y
  6 |                     fused_reshape_add_reshape_transpose_reshape_1 |     49152 |     24 |        22.7076 |       2.1646 |               51.9496 |      5 |          Y
  7 |                                          fused_reshape_divide_add |     98304 |     12 |        45.9483 |       2.1394 |               25.6734 |      5 |          Y
  8 |                                             fused_nn_fast_softmax |   4374528 |     12 |      1350.8093 |       3.2384 |               38.8614 |    125 |          Y
  9 |                                                     fused_reshape |         1 |     12 |         0.0005 |       2.1827 |               26.1925 |      5 |          Y
 10 |                                             fused_nn_batch_matmul |   6291456 |     24 |      1427.9791 |       4.4058 |              105.7403 |    960 |          Y
 11 |                                   fused_reshape_transpose_reshape |         1 |     12 |         0.0005 |       2.1435 |               25.7225 |      5 |          Y
 12 |                                                    fused_nn_dense |  75497472 |     48 |      3810.7617 |      19.8116 |              950.9591 |   7424 |
 13 |                                                   fused_reshape_1 |         1 |     24 |         0.0005 |       2.1412 |               51.3892 |      5 |          Y
 14 |                                                  fused_nn_dense_1 | 301989888 |     12 |      6289.6575 |      48.0137 |              576.1647 |   4544 |
 15 | fused_reshape_add_multiply_fast_erf_multiply_add_multiply_reshape |  15532032 |     12 |      4535.0678 |       3.4249 |               41.0985 |      5 |
 16 |                                                  fused_nn_dense_2 | 301989888 |     12 |      4206.6203 |      71.7892 |              861.4703 |   6656 |
 17 |                                             fused_reshape_add_add |     98304 |     24 |        39.9511 |       2.4606 |               59.0547 |      5 |          Y
 18 |                       fused_subtract_add_sqrt_divide_multiply_add |    196672 |     25 |        72.8936 |       2.6981 |               67.4517 |      5 |          Y
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total trials: 20046
Total latency (us): 3032.42

CC @masahi

@masahi masahi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix!

@kparzysz-quic kparzysz-quic merged commit 6f0017a into apache:main Apr 2, 2022
pfk-beta pushed a commit to pfk-beta/tvm that referenced this pull request Apr 11, 2022
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants