Skip to content

Update OS feedback on rewrite #7

Closed
mhoppa wants to merge 88 commits into
rob05c:to-go-ds-safefrom
mhoppa:feature/rewrite_ds_id_safe
Closed

Update OS feedback on rewrite #7
mhoppa wants to merge 88 commits into
rob05c:to-go-ds-safefrom
mhoppa:feature/rewrite_ds_id_safe

Conversation

@mhoppa

@mhoppa mhoppa commented Feb 4, 2020

Copy link
Copy Markdown

Updated OS feedback from OS PR on rewrite. Also needed to merge in master that is why the PR looks so big.

lactose and others added 30 commits December 30, 2019 13:34
…ers (apache#4191)

* initial sketching of API federations-federation-resolvers rewrite

* Adds federations/{id}/federation_resolvers lib code

* Fixes compile issues with federation federation resolvers API

* Fixes logic issues in assignment of federation federation_resolvers

* initial apiv14 test skeleton

* fixes ID issue with fedfed resolvers POST

* Adds replace logic to assign fedfed resolvers

* style changes, clean up for fedfed resolvers apis

* adds working tests for federation_federation_resolvers

* clean up comments

* Fixes inconsistency in naming of federationfederationresolver

(cherry picked from commit 91cf59c)

* Removes debug statements, clean up lint warnings

* Fixes user/system error mix up. Adds changelog entry

* Removes unused struct from federation federation resolver lib

* changes to make FFR API test less brittle

* Adds back .golangci.yml -- accidentally removed.

* Fixes comments, test vulnerabilities, error values for federations/{id}/federation_resolvers endpoint

* Federation Federation Resolvers: change return badrequest to notfound

* Fixes failing tests in cdnfederations

* Fixes discrepancies between perl/go for federation_federation_resolvers GET
* Fixed CiaB build issue

* Removed commented-out lines
…s-perl (apache#4247)

* Copy files from a previous build stage instead of trafficops-perl

* Remove unused traffic_ops_golang binary from trafficops-perl to save 18M

* Avoid RPM spec %post warning warnings about /etc/cron.d not existing

* Ensure that $TRAFFIC_OPS_RPM is defined

* Reduce trafficops-go to a single build stage

* Make TRAFFIC_OPS_RPM an environment variable so trafficops-perl can inherit it

* Use trafficops-go as a base image

* Do not copy in files that trafficops-go already added

* Use COPY instead of ADD for local files

* Install Perl modules and goose after Traffic Ops RPM + dependencies

* Omit packages that are installed earlier or later

* Revert "Omit packages that are installed earlier or later"

This reverts commit 5ac650991fa001e4d0956e3f8e0b329caf198e6a.

* Revert "Install Perl modules and goose after Traffic Ops RPM + dependencies"

This reverts commit 6af012e6c6fce6c57e03d0aaccf11ed35b06d83b.

* Revert "Use COPY instead of ADD for local files"

This reverts commit 678fdb3f2bc0f09817eff9865d6f9aadc9baaf42.

* Revert "Do not copy in files that trafficops-go already added"

This reverts commit 6b1130e78185ef6614e93263f7c13ba6f9eb5290.

* Revert "Use trafficops-go as a base image"

This reverts commit 01af85be78da13edbddc42aaa1140696369eef44.

* Revert "Make TRAFFIC_OPS_RPM an environment variable so trafficops-perl can inherit it"

This reverts commit 53c8bc5868a44115e4e226651098993cc41cddd0.

* Add dockerignores for trafficops-go and trafficops-perl Dockerfiles

* Add Apache License header
* Rewrite stats_summary from Perl to Golang

* Add API tests

* Update Changelog

* Update traffic_stats.go

* Merge conflict

* Address comments

* Fix API tests

* Match perl implementation on stats_summary last updated

* Fix docs
* Add waits on generate keys buttons to show up in TP tests

* Use const instead of var
* Block DSRC assignment when DS type is not HTTP or DNS

* Address feedback

* Block DSRC assignment in TP

* Fix API tests
* switches asn calls to use query params

* adds todos for update and delete calls

* adds todos to use query params for capabilitiy calls

* adds todos to cdnservice

* proper syntax for asnservice query params

* fixes query param syntax in dsrequestservice

* removal of deprecated api call

* removes unused service call

* removes use of deprecated POST user/current/update

* adds todos as a reminder to switch out api calls when support for query params is added
* EDNS0 Client Subnet enable on DS level

* CHANGELOG updates and small updates

* fixed formatting

* updates per comments

* updates per comments

* updated unit tests
* updated TP to use API 1.5 and TO perl to use 1.5

* formatting
…the api_capability table. Doing this depends on endpoints that are to be deprecated. (apache#4236)
* Fixed up the development documentation for Traffic Ops

Removed things that are now irrelevant/no longer extant in the repo, and combined the
Perl-specific and Go-specific pages into one section.

* Removed seemingly useless file

* added definitions for (nearly) all of the environment variables used by the TO API/client integration tests

The only one I left out is INSECURE, which has literally no effect.

* Finished documenting integration test configuration and started working on guidelines for new endpoints

* Finished documenting guidelines for new API endpoints

* Added text roles for linking to Go documentation

Also added associated documentation for those roles, so if you wanna
know what they are and what they do, read it.

* Added Go plugin documentation

* Updated 'Developer Environment' section

* Removed now-useless Go-specific dev docs section

* Fix minor typo

* Removed file that has been deleted from RPM spec file list

* Removed now-deleted directory from rsync list
* fixed incorrect required roles documented for /deliveryservices/{{ID}}/servers/eligible

This bug also in 4.0RC1

* fixed incorrect required roles documented for /deliveryservices/{{ID}}/servers

This bug also in 4.0RC1

* fixed incorrect required roles documented for /deliveryservices/{{ID}}/safe

This bug also in 4.0RC1

* fixed incorrect required roles documented for /deliveryservices/{{ID}}/routing

This bug also in 4.0RC1

* fixed incorrect required roles documented for /deliveryservices/{{ID}}/health

This bug also in 4.0RC1

* fixed incorrect required roles documented for /deliveryservices/{{ID}}/capacity

This bug also in 4.0RC1

* Removed erroneous caveat about tenancy and role interactions
Will make debugging errors much easier.
* Fix DSRC error response

* Updated unit tests for new dsrc resp
Fixes atstccfg to not log errors for any_map and steering DSes which
are unrelated to the DS of the config file.
apache#4256)

* Fix atscfg meta to omit nonexistent ds files

This fixes atscfg to omit files with location parameters for config
files, hdr_rw_, regex_remap_, url_sig_, and uri_signing_ for DSes
which don't exist (typically after deleting a DS but forgetting to
delete the corresponding location Parameters).

This is important to make atstccfg work because without it, ORT will
request url_sig_ files for DSes which don't exist, and the API
endpoint to get the keys will fail. Because atstccfg uses the API,
there's no reasonable way to make it work, except to fix the meta
config to omit those nonexistent files.

In addition to nonexistent files, this also omits files for DSes
not assigned to the server. There won't even be remap lines for
those DSes anyway, so it's better and safer to omit the ancillary
files altogether.

* Fix TO API test for meta cfg to set DSS, real DS
* Fix incorrect VNC information

* Fix wording to make sense
* Clarify Traffic Portal development instructions

* Further clarifications

* Fix warnings

* Use tabs
…ruct (apache#4303)

* Updated GetDeliveryServiceByXMLID function to return DS nullable struct

* Fix api tests that use the GetDeliveryServiceByXMLID function

* Added GetDeliveryServiceByXMLIDNullable function

* Reset test changes

* Added these changes to the CHANGELOG
…ache#4296)

* Fix ip_allow config generation to include rascal servers

* Fix ip_allow generation in atstccfg

* Add API tests for ip_allow

* Make tests more robust
mhoppa and others added 24 commits January 29, 2020 08:50
* Deprecate riak stats

* Update Riak.pm
* Email address cannot be empty any longer on user/current PUT

* Updated documentation

* Added constraint to client/api integration tests

* Fixed segfault with null email addresses
* Rewrite stats_summary from Perl to Golang

* Add API tests

* Update Changelog

* Update traffic_stats.go

* Merge conflict

* Address comments

* Add POST Summary stats

* Move stats_summary POST to 1.5

* Fix docs issues

* Actually adress comments for docs

* Update changelog
* Deprecate GET jobs/{id}

* Update docs
* Initial implementation of to_extension POST rewrite

* Rewrite to_extension POST

* remove commented out code

* Remove non CHECK_EXTENSION create support as its broken in perl

* Deprecate/Rewrite to_extension DELETE

* Minor cleanup

* Docs fixes
* Deprecate stats_summary/create

* Add changelog / remove docs as no need to add deprecated docs

* Update deprecation alt

* Update StatsSummary.pm
Fixes a misordered SQL scan, which caused the Server Status to be
missing, causing the config gen to think there were no online servers,
and thus not create the max_origin_connections entry.
* Deprecate /cdns/configs

* Address documentation comments
* Fix API tests for header rewrite

* Better approach :)
* Added deprecation notices to endpoint responses

* Added deprecation notices to docs, updated examples

* Changed helper name to reduce collisions

* Fixed misspelled 'parameter' as 'paramter'
* Deprecate ds/:id/state

* Update Deliveryservice.pm

* Fix documentation for endpoint
apache#4343)

* traffic_ops_ort: fix for syncds having too many header rewrite false positives

* traffic_ops_ort.pl: allow all files with changes to run through the diff_file_lines function
* Initial implementation of to_extension POST rewrite

* Rewrite to_extension POST

* remove commented out code

* Remove non CHECK_EXTENSION create support as its broken in perl

* missing license

* Update toextension.go

* Update db_helpers.go

* Update toextension.go

* Rewrite GET to_extensions

* Fix duplicate in routes

* Add plugins to to_extensions response

* Changelog update

* Address feedback
Fixes the serverchecks and to_extensions endpoints to always return
numbers, as they should.

Apparently, certain versions of Perl or its libraries sometimes turn
the numbers into strings. This forces it to always be a number.

These two endpoints are specifically used in the API tests. This
makes the API tests pass correctly.

We should probably make all numbers in all endpoints do this; but
because Perl is going away, that may not be worth it. But these two
at least let devs verify all API tests pass, instead of potentially
having to ignore failures and hope their code changes didn't cause
other failures in the same tests.
* Deprecated /types/trimmed, fixed routing issue

* Added deprecation notice to documentation

* Updated CHANGELOG.md

* Added alternative route to deprecation warning

* typo fix
* Added deprecation notices to /user/current/jobs

* Updated examples in documentation

* removed commented-out code
* deprecating /deliveryservice_user endpoint

* updated changelog
* Add TO Go cdns/routing

* Parallelize pulling data from TRs

* Fix changelog

* Licenses and comments

* Update routes.go

* Update routes.go

* Update documentation

Co-authored-by: Robert Butts <rob05c@users.noreply.github.com>
@mhoppa

mhoppa commented Feb 4, 2020

Copy link
Copy Markdown
Author

3f780f3 the actual changes can be seen

@mhoppa mhoppa closed this Mar 4, 2020
@mhoppa mhoppa deleted the feature/rewrite_ds_id_safe branch March 4, 2020 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.