From d4d655a1b7cc83cf6fedfd52af5cdad4a894ebbf Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Mon, 18 Nov 2019 17:47:09 -0700 Subject: [PATCH 01/11] Rewrite stats_summary from Perl to Golang --- docs/source/api/stats_summary.rst | 175 ++++++++++++++++++ lib/go-tc/stats_summary.go | 29 +-- traffic_ops/client/stats_summary.go | 126 +++++-------- .../traffic_ops_golang/routing/routes.go | 3 + .../trafficstats/stats_summary.go | 127 +++++++++++++ traffic_stats/traffic_stats.go | 23 +-- .../github.com/apache/trafficcontrol/lib | 1 - .../apache/trafficcontrol/traffic_ops/client | 1 - 8 files changed, 373 insertions(+), 112 deletions(-) create mode 100644 docs/source/api/stats_summary.rst create mode 100644 traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go delete mode 120000 traffic_stats/vendor/github.com/apache/trafficcontrol/lib delete mode 120000 traffic_stats/vendor/github.com/apache/trafficcontrol/traffic_ops/client diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst new file mode 100644 index 0000000000..48c9b90cba --- /dev/null +++ b/docs/source/api/stats_summary.rst @@ -0,0 +1,175 @@ +.. +.. +.. Licensed under the Apache License, Version 2.0 (the "License"); +.. you may not use this file except in compliance with the License. +.. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. +.. + +.. _to-api-stats-summary: + +***************** +``stats_summary`` +***************** + +``GET`` +======= +Either retrieve a list of summary stats or the timestamp of the latest recorded stats summary. + +What is returned is driven by the query parameter ``lastSummaryDate``. + +If the parameter is set it will return an object with the latest timestamp, else an array of summary stats will be returned. + +:Auth. Required: Yes +:Roles Required: None +:Response Type: Array or Object + +Request Structure +----------------- + +Summary Stats +""""""""""""" + +.. table:: Request Query Parameters + + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | Name | Required | Description | + +=====================+==========+======================================================================================================+ + | deliveryServiceName | no | Return only summary stats that were reported for :term:`Delivery Service` with the given name | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | cdnName | no | Return only summary stats that were reported for CDN with the given name | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | statName | no | Return only summary stats that were reported for given stat name | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | orderby | no | Choose the ordering of the results - can only be one of deliveryServiceName, statName or cdnName | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | sortOrder | no | Changes the order of sorting. Either ascending (default or "asc") or | + | | | descending ("desc") | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | limit | no | Choose the maximum number of results to return | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | offset | no | The number of results to skip before beginning to return results. Must use in conjunction with limit | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + | page | no | Return the n\ :sup:`th` page of results, where "n" is the value of this parameter, pages are | + | | | ``limit`` long and the first page is 1. If ``offset`` was defined, this query parameter has no | + | | | effect. ``limit`` must be defined to make use of ``page``. | + +---------------------+----------+------------------------------------------------------------------------------------------------------+ + +.. code-block:: http + :caption: Request Example + + GET /api/1.4/stats_summary HTTP/1.1 + Host: trafficops.infra.ciab.test + User-Agent: curl/7.47.0 + Accept: */* + Cookie: mojolicious=... + +Last Updated Summary Stat +"""""""""""""""""""""""""" + +.. table:: Request Query Parameters + + +-----------------+----------+---------------------------------------------------+ + | Name | Required | Description | + +=================+==========+===================================================+ + | statName | no | Get lastest updated date for the given stat | + +-----------------+----------+---------------------------------------------------+ + | lastSummaryDate | yes | Tells route to get only lastest updated timestamp | + +-----------------+----------+---------------------------------------------------+ + +.. code-block:: http + :caption: Request Example + + GET /api/1.4/stats_summary?lastSummaryDate=true HTTP/1.1 + Host: trafficops.infra.ciab.test + User-Agent: curl/7.47.0 + Accept: */* + Cookie: mojolicious=... + +Response Structure +------------------ + +Summary Stats +""""""""""""" + +:cdnName: CDN name summary stat was taken for +:deliveryServiceName: :term:`Delivery Service` name summary stat was taken for +:statName: Stat name summary stat represents +:statValue: Summary stat value +:summaryTime: Timestamp of summary, in :rfc:`3339` format +:statDate: Date stat was taken + +.. code-block:: http + :caption: Response Example + + HTTP/1.1 200 OK + Access-Control-Allow-Credentials: true + Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Set-Cookie, Cookie + Access-Control-Allow-Methods: POST,GET,OPTIONS,PUT,DELETE + Access-Control-Allow-Origin: * + Content-Type: application/json + Set-Cookie: mojolicious=...; Path=/; Expires=Mon, 18 Nov 2019 17:40:54 GMT; Max-Age=3600; HttpOnly + Whole-Content-Sha512: dHNip9kpTGGS1w39/fWcFehNktgmXZus8XaufnmDpv0PyG/3fK/KfoCO3ZOj9V74/CCffps7doEygWeL/xRtKA== + X-Server-Name: traffic_ops_golang/ + Date: Mon, 10 Dec 2018 20:56:59 GMT + Content-Length: 150 + + { "response": [ + { + "cdnName": "CDN-in-a-Box", + "deliveryServiceName": "all", + "statName": "daily_maxgbps", + "statValue": 5, + "summaryTime": "2019-11-19T00:04:06Z", + "statDate": "2019-11-18T00:00:00Z" + }, + { + "cdnName": "CDN-in-a-Box", + "deliveryServiceName": "all", + "statName": "daily_maxgbps", + "statValue": 3, + "summaryTime": "2019-11-18T00:04:06Z", + "statDate": "2019-11-17T00:00:00Z" + }, + { + "cdnName": "CDN-in-a-Box", + "deliveryServiceName": "all", + "statName": "daily_bytesserved", + "statValue": 1000, + "summaryTime": "2019-11-19T00:04:06Z", + "statDate": "2019-11-18T00:00:00Z" + }, + ]} + +Last Updated Summary Stat +""""""""""""""""""""""""" + +:summaryTime: Timestamp of the last updated summary, in :rfc:`3339` format + +.. code-block:: http + :caption: Response Example + + HTTP/1.1 200 OK + Access-Control-Allow-Credentials: true + Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Set-Cookie, Cookie + Access-Control-Allow-Methods: POST,GET,OPTIONS,PUT,DELETE + Access-Control-Allow-Origin: * + Content-Type: application/json + Set-Cookie: mojolicious=...; Path=/; Expires=Mon, 18 Nov 2019 17:40:54 GMT; Max-Age=3600; HttpOnly + Whole-Content-Sha512: dHNip9kpTGGS1w39/fWcFehNktgmXZus8XaufnmDpv0PyG/3fK/KfoCO3ZOj9V74/CCffps7doEygWeL/xRtKA== + X-Server-Name: traffic_ops_golang/ + Date: Mon, 10 Dec 2018 20:56:59 GMT + Content-Length: 150 + + { "response": + { + "summaryTime": "2019-11-19T00:04:06Z" + } + } \ No newline at end of file diff --git a/lib/go-tc/stats_summary.go b/lib/go-tc/stats_summary.go index ad3f3aaf50..d9b6ae5666 100644 --- a/lib/go-tc/stats_summary.go +++ b/lib/go-tc/stats_summary.go @@ -1,5 +1,7 @@ package tc +import "time" + /* * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file @@ -26,18 +28,21 @@ type StatsSummaryResponse struct { // StatsSummary ... type StatsSummary struct { - CDNName string `json:"cdnName"` - DeliveryService string `json:"deliveryServiceName"` - StatName string `json:"statName"` - StatValue string `json:"statValue"` - SummaryTime string `json:"summaryTime"` - StatDate string `json:"statDate"` + ID int `json:"-" db:"id"` + CDNName string `json:"cdnName" db:"cdn_name"` + DeliveryService string `json:"deliveryServiceName" db:"deliveryservice_name"` + StatName string `json:"statName" db:"stat_name"` + StatValue float64 `json:"statValue" db:"stat_value"` + SummaryTime time.Time `json:"summaryTime" db:"summary_time"` + StatDate time.Time `json:"statDate" db:"stat_date"` +} + +// StatsSummaryLastUpdated ... +type StatsSummaryLastUpdated struct { + SummaryTime time.Time `json:"summaryTime" db:"summary_time"` } -// LastUpdated ... -type LastUpdated struct { - Version string `json:"version"` - Response struct { - SummaryTime string `json:"summaryTime"` - } `json:"response"` +// StatsSummaryLastUpdatedResponse ... +type StatsSummaryLastUpdatedResponse struct { + Response StatsSummaryLastUpdated `json:"response"` } diff --git a/traffic_ops/client/stats_summary.go b/traffic_ops/client/stats_summary.go index 8a6c909079..ef965a75c1 100644 --- a/traffic_ops/client/stats_summary.go +++ b/traffic_ops/client/stats_summary.go @@ -15,111 +15,69 @@ package client import ( "encoding/json" "fmt" - "strconv" + "net" + "net/url" tc "github.com/apache/trafficcontrol/lib/go-tc" ) -// SummaryStats ... -// Deprecated: use GetSummaryStats -func (to *Session) SummaryStats(cdn string, deliveryService string, statName string) ([]tc.StatsSummary, error) { - ss, _, err := to.GetSummaryStats(cdn, deliveryService, statName) - return ss, err -} +const ( + basePathSummaryStats = apiBase + "/stats_summary" +) -func (to *Session) GetSummaryStats(cdn string, deliveryService string, statName string) ([]tc.StatsSummary, ReqInf, error) { - var queryParams []string - if len(cdn) > 0 { - queryParams = append(queryParams, fmt.Sprintf("cdnName=%s", cdn)) - } - if len(deliveryService) > 0 { - queryParams = append(queryParams, fmt.Sprintf("deliveryServiceName=%s", deliveryService)) +// GetSummaryStats gets a list of summary stats with the ability to filter on cdn,deliveryService and/or stat +func (to *Session) GetSummaryStats(cdn, deliveryService, statName *string) (tc.StatsSummaryResponse, ReqInf, error) { + resp := tc.StatsSummaryResponse{} + + param := url.Values{} + if cdn != nil { + param.Add("cdnName", *cdn) } - if len(statName) > 0 { - queryParams = append(queryParams, fmt.Sprintf("statName=%s", statName)) + if deliveryService != nil { + param.Add("deliveryServiceName", *deliveryService) } - queryURL := "/api/1.2/stats_summary.json" - queryParamString := "?" - if len(queryParams) > 0 { - for i, param := range queryParams { - if i == 0 { - queryParamString += param - } else { - queryParamString += fmt.Sprintf("&%s", param) - } - } - queryURL += queryParamString + if statName != nil { + param.Add("statName", *statName) } - resp, remoteAddr, err := to.request("GET", queryURL, nil) - reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr} - if err != nil { - return nil, reqInf, err + url := basePathSummaryStats + if len(param) > 0 { + url = fmt.Sprintf("%s?%s", basePathSummaryStats, param.Encode()) } - defer resp.Body.Close() - - var data tc.StatsSummaryResponse - if err := json.NewDecoder(resp.Body).Decode(&data); err != nil { - return nil, reqInf, err + reqInf, err := get(to, url, &resp) + if err != nil { + return resp, reqInf, err } - - return data.Response, reqInf, nil + return resp, reqInf, nil } -// SummaryStatsLastUpdated ... -// Deprecated: use GetSummaryStatsLastUpdated -func (to *Session) SummaryStatsLastUpdated(statName string) (string, error) { - s, _, err := to.GetSummaryStatsLastUpdated(statName) - return s, err -} +// GetSummaryStatsLastUpdated time of the last summary for a given stat +func (to *Session) GetSummaryStatsLastUpdated(statName *string) (tc.StatsSummaryLastUpdatedResponse, ReqInf, error) { + resp := tc.StatsSummaryLastUpdatedResponse{} -func (to *Session) GetSummaryStatsLastUpdated(statName string) (string, ReqInf, error) { - queryURL := "/api/1.2/stats_summary.json?lastSummaryDate=true" - if len(statName) > 0 { - queryURL += fmt.Sprintf("?statName=%s", statName) + param := url.Values{} + param.Add("lastSummaryDate", "true") + if statName != nil { + param.Add("statName", *statName) } + url := fmt.Sprintf("%s?%s", basePathSummaryStats, param.Encode()) - resp, remoteAddr, err := to.request("GET", queryURL, nil) - reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr} + reqInf, err := get(to, url, &resp) if err != nil { - return "", reqInf, err + return resp, reqInf, err } - defer resp.Body.Close() - - var data tc.LastUpdated - if err := json.NewDecoder(resp.Body).Decode(&data); err != nil { - return "", reqInf, err - } - - if len(data.Response.SummaryTime) > 0 { - return data.Response.SummaryTime, reqInf, nil - } - t := "1970-01-01 00:00:00" - return t, reqInf, nil + return resp, reqInf, nil } -// AddSummaryStats ... -// Deprecated: use DoAddSummaryStats -func (to *Session) AddSummaryStats(statsSummary tc.StatsSummary) error { - _, err := to.DoAddSummaryStats(statsSummary) - return err -} -func (to *Session) DoAddSummaryStats(statsSummary tc.StatsSummary) (ReqInf, error) { - reqBody, err := json.Marshal(statsSummary) - if err != nil { - return ReqInf{}, err - } - - url := "/api/1.2/stats_summary/create" - resp, remoteAddr, err := to.request("POST", url, reqBody) +// CreateSummaryStats creates a stats summary +func (to *Session) CreateSummaryStats(statsSummary tc.StatsSummary) (tc.Alerts, ReqInf, error) { + var alerts tc.Alerts + var remoteAddr net.Addr reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr} + reqBody, err := json.Marshal(statsSummary) if err != nil { - return reqInf, err - } - defer resp.Body.Close() - if resp.StatusCode != 200 { - err := fmt.Errorf("Response code = %s and Status = %s", strconv.Itoa(resp.StatusCode), resp.Status) - return reqInf, err + return tc.Alerts{}, reqInf, err } - return reqInf, nil + reqInf, err = post(to, fmt.Sprintf("%s/%s", basePathSummaryStats, "create"), reqBody, &alerts) + return alerts, reqInf, err } diff --git a/traffic_ops/traffic_ops_golang/routing/routes.go b/traffic_ops/traffic_ops_golang/routing/routes.go index bf9f21248f..a7b7eec752 100644 --- a/traffic_ops/traffic_ops_golang/routing/routes.go +++ b/traffic_ops/traffic_ops_golang/routing/routes.go @@ -528,6 +528,9 @@ func Routes(d ServerData) ([]Route, []RawRoute, http.Handler, error) { {1.1, http.MethodPut, `steering/{deliveryservice}/targets/{target}/?(\.json)?$`, api.UpdateHandler(&steeringtargets.TOSteeringTargetV11{}), auth.PrivLevelSteering, Authenticated, nil}, {1.1, http.MethodDelete, `steering/{deliveryservice}/targets/{target}/?(\.json)?$`, api.DeleteHandler(&steeringtargets.TOSteeringTargetV11{}), auth.PrivLevelSteering, Authenticated, nil}, + // Stats Summary + {1.1, http.MethodGet, `stats_summary/?(\.json)?$`, trafficstats.GetStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil}, + //Pattern based consistent hashing endpoint {1.4, http.MethodPost, `consistenthash/?$`, consistenthash.Post, auth.PrivLevelReadOnly, Authenticated, nil}, diff --git a/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go b/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go new file mode 100644 index 0000000000..61f17a1f10 --- /dev/null +++ b/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go @@ -0,0 +1,127 @@ +package trafficstats + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import ( + "fmt" + "net/http" + + "github.com/apache/trafficcontrol/lib/go-tc" + "github.com/apache/trafficcontrol/lib/go-util" + "github.com/jmoiron/sqlx" + + "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api" + "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/dbhelpers" +) + +// GetStatsSummary handler for getting stats summaries +func GetStatsSummary(w http.ResponseWriter, r *http.Request) { + inf, userErr, sysErr, errCode := api.NewInfo(r, []string{}, []string{}) + if userErr != nil || sysErr != nil { + api.HandleErr(w, r, inf.Tx.Tx, errCode, userErr, sysErr) + return + } + defer inf.Close() + + lastSummaryDateStr := inf.Params["lastSummaryDate"] + if len(lastSummaryDateStr) != 0 { // Perl only checked for existence of query param + getLastSummaryDate(w, r, inf) + return + } + + getStatsSummary(w, r, inf) + return +} + +func getLastSummaryDate(w http.ResponseWriter, r *http.Request, inf *api.APIInfo) { + queryParamsToSQLCols := map[string]dbhelpers.WhereColumnInfo{ + "statName": dbhelpers.WhereColumnInfo{"stat_name", nil}, + } + where, _, _, queryValues, errs := dbhelpers.BuildWhereAndOrderByAndPagination(inf.Params, queryParamsToSQLCols) + if len(errs) > 0 { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, util.JoinErrs(errs)) + return + } + query := selectQuery() + where + " ORDER BY summary_time DESC" + statsSummaries, err := queryStatsSummary(inf.Tx, query, queryValues) + if err != nil { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, err) + return + } + if len(statsSummaries) < 1 { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusBadRequest, fmt.Errorf("no stats summary available to grab last updated time"), nil) + return + } + resp := tc.StatsSummaryLastUpdated{ + SummaryTime: statsSummaries[0].SummaryTime, + } + api.WriteResp(w, r, resp) +} + +func getStatsSummary(w http.ResponseWriter, r *http.Request, inf *api.APIInfo) { + queryParamsToSQLCols := map[string]dbhelpers.WhereColumnInfo{ + "statName": dbhelpers.WhereColumnInfo{"stat_name", nil}, + "cdnName": dbhelpers.WhereColumnInfo{"cdn_name", nil}, + "deliveryServiceName": dbhelpers.WhereColumnInfo{"deliveryservice_name", nil}, + } + where, orderBy, pagination, queryValues, errs := dbhelpers.BuildWhereAndOrderByAndPagination(inf.Params, queryParamsToSQLCols) + if len(errs) > 0 { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, util.JoinErrs(errs)) + return + } + query := selectQuery() + where + orderBy + pagination + statsSummaries, err := queryStatsSummary(inf.Tx, query, queryValues) + if err != nil { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, err) + return + } + + api.WriteResp(w, r, statsSummaries) +} + +func queryStatsSummary(tx *sqlx.Tx, q string, queryValues map[string]interface{}) ([]tc.StatsSummary, error) { + rows, err := tx.NamedQuery(q, queryValues) + if err != nil { + return nil, fmt.Errorf("querying stats summary: %v", err) + } + defer rows.Close() + + statsSummaries := []tc.StatsSummary{} + for rows.Next() { + s := tc.StatsSummary{} + if err = rows.StructScan(&s); err != nil { + return nil, fmt.Errorf("scanning stats summary: %v", err) + } + statsSummaries = append(statsSummaries, s) + } + return statsSummaries, nil +} + +func selectQuery() string { + return `SELECT +id, +cdn_name, +deliveryservice_name, +stat_name, +stat_value, +summary_time, +stat_date +FROM stats_summary` +} diff --git a/traffic_stats/traffic_stats.go b/traffic_stats/traffic_stats.go index e3bd54e6c0..c11ce72c8a 100644 --- a/traffic_stats/traffic_stats.go +++ b/traffic_stats/traffic_stats.go @@ -316,9 +316,9 @@ func calcDailyMaxGbps(client influx.Client, bp influx.BatchPoints, startTime tim statsSummary.CDNName = cdn statsSummary.DeliveryService = "all" statsSummary.StatName = "daily_maxgbps" - statsSummary.StatValue = strconv.FormatFloat(value, 'f', 2, 64) - statsSummary.SummaryTime = time.Now().Format(time.RFC3339) - statsSummary.StatDate = statTime.Format("2006-01-02") + statsSummary.StatValue = value + statsSummary.SummaryTime = time.Now() + statsSummary.StatDate = statTime go writeSummaryStats(config, statsSummary) //write to influxdb @@ -376,9 +376,9 @@ func calcDailyBytesServed(client influx.Client, bp influx.BatchPoints, startTime statsSummary.CDNName = cdn statsSummary.DeliveryService = "all" statsSummary.StatName = "daily_bytesserved" - statsSummary.StatValue = strconv.FormatFloat(bytesServedTB, 'f', 2, 64) - statsSummary.SummaryTime = time.Now().Format(time.RFC3339) - statsSummary.StatDate = startTime.Format("2006-01-02") + statsSummary.StatValue = bytesServedTB + statsSummary.SummaryTime = time.Now() + statsSummary.StatDate = time.Now() go writeSummaryStats(config, statsSummary) //write to Influxdb tags := map[string]string{"cdn": cdn, "deliveryservice": "all"} @@ -422,7 +422,7 @@ func writeSummaryStats(config StartupConfig, statsSummary tc.StatsSummary) { log.Error(newErr) return } - err = to.AddSummaryStats(statsSummary) + _, _, err = to.CreateSummaryStats(statsSummary) if err != nil { log.Error(err) } @@ -502,16 +502,11 @@ func getToData(config StartupConfig, init bool, configChan chan RunningConfig) { } } - lastSummaryTimeStr, err := to.SummaryStatsLastUpdated("daily_maxgbps") + lastSummaryTimeResponse, _, err := to.GetSummaryStatsLastUpdated("daily_maxgbps") if err != nil { errHndlr(err, ERROR) } else { - lastSummaryTime, err := time.Parse("2006-01-02 15:04:05-07", lastSummaryTimeStr) - if err != nil { - log.Error("Error parsing lastSummaryTime: " + err.Error()) - } else { - runningConfig.LastSummaryTime = lastSummaryTime - } + runningConfig.LastSummaryTime = lastSummaryTimeResponse.Response.SummaryTime } configChan <- runningConfig diff --git a/traffic_stats/vendor/github.com/apache/trafficcontrol/lib b/traffic_stats/vendor/github.com/apache/trafficcontrol/lib deleted file mode 120000 index c2c14b9959..0000000000 --- a/traffic_stats/vendor/github.com/apache/trafficcontrol/lib +++ /dev/null @@ -1 +0,0 @@ -../../../../../lib \ No newline at end of file diff --git a/traffic_stats/vendor/github.com/apache/trafficcontrol/traffic_ops/client b/traffic_stats/vendor/github.com/apache/trafficcontrol/traffic_ops/client deleted file mode 120000 index 205510b118..0000000000 --- a/traffic_stats/vendor/github.com/apache/trafficcontrol/traffic_ops/client +++ /dev/null @@ -1 +0,0 @@ -../../../../../../traffic_ops/client \ No newline at end of file From 050236200aa0e4dd42eb9b31b1f512b406c40ee0 Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Mon, 2 Dec 2019 17:01:23 -0700 Subject: [PATCH 02/11] Add API tests --- docs/source/api/stats_summary.rst | 2 +- lib/go-tc/stats_summary.go | 39 +++- .../testing/api/v14/stats_summary_test.go | 186 ++++++++++++++++++ traffic_ops/testing/api/v14/tc-fixtures.json | 18 ++ .../testing/api/v14/traffic_control_test.go | 1 + traffic_stats/traffic_stats.go | 4 +- 6 files changed, 247 insertions(+), 3 deletions(-) create mode 100644 traffic_ops/testing/api/v14/stats_summary_test.go diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst index 48c9b90cba..07612d8d9e 100644 --- a/docs/source/api/stats_summary.rst +++ b/docs/source/api/stats_summary.rst @@ -145,7 +145,7 @@ Summary Stats "statValue": 1000, "summaryTime": "2019-11-19T00:04:06Z", "statDate": "2019-11-18T00:00:00Z" - }, + } ]} Last Updated Summary Stat diff --git a/lib/go-tc/stats_summary.go b/lib/go-tc/stats_summary.go index d9b6ae5666..c82db4d06f 100644 --- a/lib/go-tc/stats_summary.go +++ b/lib/go-tc/stats_summary.go @@ -1,6 +1,9 @@ package tc -import "time" +import ( + "encoding/json" + "time" +) /* * Licensed to the Apache Software Foundation (ASF) under one @@ -21,6 +24,8 @@ import "time" * under the License. */ +const dateFormat = "2006-01-02" + // StatsSummaryResponse ... type StatsSummaryResponse struct { Response []StatsSummary `json:"response"` @@ -37,6 +42,38 @@ type StatsSummary struct { StatDate time.Time `json:"statDate" db:"stat_date"` } +// UnmarshalJSON Customized Unmarshal to force date format on statDate +func (ss *StatsSummary) UnmarshalJSON(data []byte) error { + type Alias StatsSummary + resp := struct { + StatDate string `json:"statDate"` + *Alias + }{ + Alias: (*Alias)(ss), + } + err := json.Unmarshal(data, &resp) + if err != nil { + return err + } + ss.StatDate, err = time.Parse(dateFormat, resp.StatDate) + if err != nil { + return err + } + return nil +} + +// UnmarshalJSON Customized Marshal to force date format on statDate +func (ss StatsSummary) MarshalJSON() ([]byte, error) { + type Alias StatsSummary + return json.Marshal(&struct { + StatDate string `json:"statDate"` + Alias + }{ + StatDate: ss.StatDate.Format(dateFormat), + Alias: (Alias)(ss), + }) +} + // StatsSummaryLastUpdated ... type StatsSummaryLastUpdated struct { SummaryTime time.Time `json:"summaryTime" db:"summary_time"` diff --git a/traffic_ops/testing/api/v14/stats_summary_test.go b/traffic_ops/testing/api/v14/stats_summary_test.go new file mode 100644 index 0000000000..547af03b4e --- /dev/null +++ b/traffic_ops/testing/api/v14/stats_summary_test.go @@ -0,0 +1,186 @@ +package v14 + +/* + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +import ( + "fmt" + "testing" + "time" + + "github.com/apache/trafficcontrol/lib/go-tc" + "github.com/apache/trafficcontrol/lib/go-util" +) + +var ( + testStatsSummaries []tc.StatsSummary + latestTime time.Time +) + +const dateFormat = "2006-01-02" + +func TestStatsSummary(t *testing.T) { + WithObjs(t, []TCObj{}, func() { + testStatsSummaries = []tc.StatsSummary{} + latestTime = time.Now().Truncate(time.Second).UTC() + CreateTestStatsSummaries(t) + GetTestStatsSummaries(t) + GetTestStatsSummariesLastUpdated(t) + }) +} + +func CreateTestStatsSummaries(t *testing.T) { + tmpTime := latestTime + for _, ss := range testData.StatsSummaries { + ss.SummaryTime = tmpTime + timeStamp, err := time.Parse(dateFormat, tmpTime.Format(dateFormat)) + if err != nil { + t.Errorf("error getting timestamp at beginning of the day: %v", err) + } + + ss.StatDate = timeStamp + + _, _, err = TOSession.CreateSummaryStats(ss) + if err != nil { + t.Errorf("creating stats_summary %v: %v", ss.StatName, err) + } + + tmpTime = tmpTime.AddDate(0, 0, -1) + + testStatsSummaries = append(testStatsSummaries, ss) + } +} + +func GetTestStatsSummaries(t *testing.T) { + var testCases = []struct { + description string + stat *string + cdn *string + ds *string + expectedStatsSummaries []tc.StatsSummary + }{ + { + description: "get all summary stats", + expectedStatsSummaries: testStatsSummaries, + }, + { + description: "non-existant stat name", + stat: util.StrPtr("bogus"), + }, + { + description: "non-existant ds name", + ds: util.StrPtr("bogus"), + }, + { + description: "non-existant cdn name", + cdn: util.StrPtr("bogus"), + }, + { + description: "get stats summary by stat name", + stat: util.StrPtr("daily_bytesserved"), + expectedStatsSummaries: func() []tc.StatsSummary { + statsSummaries := []tc.StatsSummary{} + for _, ss := range testStatsSummaries { + if ss.StatName == "daily_bytesserved" { + statsSummaries = append(statsSummaries, ss) + } + } + return statsSummaries + }(), + }, + { + description: "get stats summary by cdn name", + cdn: util.StrPtr("cdn1"), + expectedStatsSummaries: testStatsSummaries, + }, + { + description: "get stats summary by ds name", + ds: util.StrPtr("all"), + expectedStatsSummaries: testStatsSummaries, + }, + } + + for _, tc := range testCases { + t.Run(tc.description, func(t *testing.T) { + tsr, _, err := TOSession.GetSummaryStats(tc.cdn, tc.ds, tc.stat) + if err != nil { + t.Fatalf("received unexpectant error %v on GET to stats_summary", err) + } + if len(tc.expectedStatsSummaries) == 0 && len(tsr.Response) != 0 { + t.Fatalf("expected to recieve no stats summaries but received %v", len(tsr.Response)) + } + for _, ess := range tc.expectedStatsSummaries { + found := false + for _, ss := range tsr.Response { + if ess == ss { + found = true + break + } + } + if !found { + t.Fatalf("expected to find stat %v in stats summary response", ess.StatName) + } + } + }) + } +} + +func GetTestStatsSummariesLastUpdated(t *testing.T) { + type testCase struct { + description string + stat *string + errExpected bool + expectedTimestamp time.Time + } + testCases := []testCase{ + testCase{ + description: "latest updated timestamp", + stat: nil, + errExpected: false, + expectedTimestamp: latestTime, + }, + testCase{ + description: "non-existant stat name", + stat: util.StrPtr("bogus"), + errExpected: true, + }, + } + for _, ss := range testStatsSummaries { + testCases = append(testCases, testCase{ + description: fmt.Sprintf("latest updated timestamp for - %v", ss.StatName), + stat: util.StrPtr(ss.StatName), + errExpected: false, + expectedTimestamp: ss.SummaryTime, + }) + } + + for _, tc := range testCases { + t.Run(tc.description, func(t *testing.T) { + tsr, _, err := TOSession.GetSummaryStatsLastUpdated(tc.stat) + if tc.errExpected && err == nil { + t.Fatalf("expected to get error on getting stats_summary latest updated timestamp but recieved nil") + } + + if !tc.errExpected && err != nil { + t.Fatalf("recieved unexpected error getting stats_summary latest updated timestamp: %v", err) + } + + if !tc.errExpected && !tsr.Response.SummaryTime.Equal(tc.expectedTimestamp) { + t.Fatalf("recieved latest timestamp %v does not match up to expected timestamp %v", tsr.Response.SummaryTime, tc.expectedTimestamp) + } + }) + } + +} diff --git a/traffic_ops/testing/api/v14/tc-fixtures.json b/traffic_ops/testing/api/v14/tc-fixtures.json index 1944193222..ba74b365ef 100644 --- a/traffic_ops/testing/api/v14/tc-fixtures.json +++ b/traffic_ops/testing/api/v14/tc-fixtures.json @@ -2533,5 +2533,23 @@ "startTime": "2100-06-19T13:57:51-06:00", "ttl": 2.1 } + ], + "stats_summaries": [ + { + "cdnName": "cdn1", + "deliveryServiceName": "all", + "statName": "daily_maxgbps", + "statValue": 5, + "summaryTime": "2019-01-01T00:00:00-06:00", + "statDate": "2019-01-01" + }, + { + "cdnName": "cdn1", + "deliveryServiceName": "all", + "statName": "daily_bytesserved", + "statValue": 1000, + "summaryTime": "2019-01-01T00:00:00-06:00", + "statDate": "2019-01-01" + } ] } diff --git a/traffic_ops/testing/api/v14/traffic_control_test.go b/traffic_ops/testing/api/v14/traffic_control_test.go index 87efcdd229..8da742aabf 100644 --- a/traffic_ops/testing/api/v14/traffic_control_test.go +++ b/traffic_ops/testing/api/v14/traffic_control_test.go @@ -45,6 +45,7 @@ type TrafficControl struct { ServerCapabilities []tc.ServerCapability `json:"serverCapabilities"` Statuses []tc.StatusNullable `json:"statuses"` StaticDNSEntries []tc.StaticDNSEntry `json:"staticdnsentries"` + StatsSummaries []tc.StatsSummary `json:"stats_summaries"` Tenants []tc.Tenant `json:"tenants"` TOExtensions []tc.TOExtensionNullable `json:"to_extensions"` Types []tc.Type `json:"types"` diff --git a/traffic_stats/traffic_stats.go b/traffic_stats/traffic_stats.go index c11ce72c8a..76c3dbdca8 100644 --- a/traffic_stats/traffic_stats.go +++ b/traffic_stats/traffic_stats.go @@ -36,6 +36,8 @@ import ( "time" "github.com/apache/trafficcontrol/lib/go-tc" + "github.com/apache/trafficcontrol/lib/go-util" + "github.com/apache/trafficcontrol/traffic_ops/client" log "github.com/cihub/seelog" influx "github.com/influxdata/influxdb/client/v2" @@ -502,7 +504,7 @@ func getToData(config StartupConfig, init bool, configChan chan RunningConfig) { } } - lastSummaryTimeResponse, _, err := to.GetSummaryStatsLastUpdated("daily_maxgbps") + lastSummaryTimeResponse, _, err := to.GetSummaryStatsLastUpdated(util.StrPtr("daily_maxgbps")) if err != nil { errHndlr(err, ERROR) } else { From 1b9ad179f4a6f14ac3f0c9d31c994545211e8833 Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Mon, 2 Dec 2019 17:05:33 -0700 Subject: [PATCH 03/11] Update Changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8cd68c91c0..663fa92b31 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - /api/1.1/federations/:id/users - /api/1.1/federations/:id/users/:userID - /api/1.1/osversions + - /api/1.1/stats_summary `GET` - To support reusing a single riak cluster connection, an optional parameter is added to riak.conf: "HealthCheckInterval". This options takes a 'Duration' value (ie: 10s, 5m) which affects how often the riak cluster is health checked. Default is currently set to: "HealthCheckInterval": "5s". - Added a new Go db/admin binary to replace the Perl db/admin.pl script which is now deprecated and will be removed in a future release. The new db/admin binary is essentially a drop-in replacement for db/admin.pl since it supports all of the same commands and options; therefore, it should be used in place of db/admin.pl for all the same tasks. From c17be6164b65a2372d69cabf9a22561c2db1279e Mon Sep 17 00:00:00 2001 From: Michael Hoppal <54542891+mhoppa@users.noreply.github.com> Date: Tue, 3 Dec 2019 11:10:48 -0700 Subject: [PATCH 04/11] Update traffic_stats.go --- traffic_stats/traffic_stats.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/traffic_stats/traffic_stats.go b/traffic_stats/traffic_stats.go index 76c3dbdca8..77139a20a8 100644 --- a/traffic_stats/traffic_stats.go +++ b/traffic_stats/traffic_stats.go @@ -380,7 +380,7 @@ func calcDailyBytesServed(client influx.Client, bp influx.BatchPoints, startTime statsSummary.StatName = "daily_bytesserved" statsSummary.StatValue = bytesServedTB statsSummary.SummaryTime = time.Now() - statsSummary.StatDate = time.Now() + statsSummary.StatDate = startTime go writeSummaryStats(config, statsSummary) //write to Influxdb tags := map[string]string{"cdn": cdn, "deliveryservice": "all"} From 2668b2110fc4d82a91d9fca4fd0366d34b7b2df9 Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Tue, 3 Dec 2019 11:40:53 -0700 Subject: [PATCH 05/11] Merge conflict --- traffic_ops/traffic_ops_golang/routing/routes.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/traffic_ops/traffic_ops_golang/routing/routes.go b/traffic_ops/traffic_ops_golang/routing/routes.go index d7615ba400..425bb20896 100644 --- a/traffic_ops/traffic_ops_golang/routing/routes.go +++ b/traffic_ops/traffic_ops_golang/routing/routes.go @@ -558,7 +558,7 @@ func Routes(d ServerData) ([]Route, []RawRoute, http.Handler, error) { {1.1, http.MethodDelete, `steering/{deliveryservice}/targets/{target}/?(\.json)?$`, api.DeleteHandler(&steeringtargets.TOSteeringTargetV11{}), auth.PrivLevelSteering, Authenticated, nil, 2088021515, noPerlBypass}, // Stats Summary - {1.1, http.MethodGet, `stats_summary/?(\.json)?$`, trafficstats.GetStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil}, + {1.1, http.MethodGet, `stats_summary/?(\.json)?$`, trafficstats.GetStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil, 380498598, perlBypass}, //Pattern based consistent hashing endpoint {1.4, http.MethodPost, `consistenthash/?$`, consistenthash.Post, auth.PrivLevelReadOnly, Authenticated, nil, 1960755076, noPerlBypass}, From 6a83b2733a67a4efd4774da6231627a079f9ebca Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Thu, 5 Dec 2019 12:09:55 -0700 Subject: [PATCH 06/11] Address comments --- docs/source/api/stats_summary.rst | 69 ++++++++++--------- lib/go-tc/stats_summary.go | 26 +++++-- traffic_ops/client/stats_summary.go | 10 +-- .../testing/api/v14/stats_summary_test.go | 16 ++--- traffic_ops/testing/api/v14/tc-fixtures.json | 2 +- .../testing/api/v14/traffic_control_test.go | 2 +- 6 files changed, 68 insertions(+), 57 deletions(-) diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst index 07612d8d9e..8bcd4497ad 100644 --- a/docs/source/api/stats_summary.rst +++ b/docs/source/api/stats_summary.rst @@ -39,28 +39,28 @@ Summary Stats .. table:: Request Query Parameters - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | Name | Required | Description | - +=====================+==========+======================================================================================================+ - | deliveryServiceName | no | Return only summary stats that were reported for :term:`Delivery Service` with the given name | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | cdnName | no | Return only summary stats that were reported for CDN with the given name | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | statName | no | Return only summary stats that were reported for given stat name | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | orderby | no | Choose the ordering of the results - can only be one of deliveryServiceName, statName or cdnName | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | sortOrder | no | Changes the order of sorting. Either ascending (default or "asc") or | - | | | descending ("desc") | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | limit | no | Choose the maximum number of results to return | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | offset | no | The number of results to skip before beginning to return results. Must use in conjunction with limit | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ - | page | no | Return the n\ :sup:`th` page of results, where "n" is the value of this parameter, pages are | - | | | ``limit`` long and the first page is 1. If ``offset`` was defined, this query parameter has no | - | | | effect. ``limit`` must be defined to make use of ``page``. | - +---------------------+----------+------------------------------------------------------------------------------------------------------+ + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | Name | Required | Description | + +=====================+==========+=======================================================================================================+ + | deliveryServiceName | no | Return only summary stats that were reported for :term:`Delivery Service` with the given display name | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | cdnName | no | Return only summary stats that were reported for CDN with the given name | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | statName | no | Return only summary stats that were reported for given stat name | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | orderby | no | Choose the ordering of the results - can only be one of deliveryServiceName, statName or cdnName | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | sortOrder | no | Changes the order of sorting. Either ascending (default or "asc") or | + | | | descending ("desc") | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | limit | no | Choose the maximum number of results to return | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | offset | no | The number of results to skip before beginning to return results. Must use in conjunction with limit | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ + | page | no | Return the n\ :sup:`th` page of results, where "n" is the value of this parameter, pages are | + | | | ``limit`` long and the first page is 1. If ``offset`` was defined, this query parameter has no | + | | | effect. ``limit`` must be defined to make use of ``page``. | + +---------------------+----------+-------------------------------------------------------------------------------------------------------+ .. code-block:: http :caption: Request Example @@ -99,12 +99,15 @@ Response Structure Summary Stats """"""""""""" -:cdnName: CDN name summary stat was taken for -:deliveryServiceName: :term:`Delivery Service` name summary stat was taken for +:cdnName: The CDN name for which the summary stat was taken for +:deliveryServiceName: The :term:`Delivery Service` display name for which the summary stat was taken for + + .. note:: If the ``deliveryServiceName`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN + :statName: Stat name summary stat represents :statValue: Summary stat value -:summaryTime: Timestamp of summary, in :rfc:`3339` format -:statDate: Date stat was taken +:summaryTime: Timestamp of summary, in an ISO-like format +:statDate: Date stat was taken, in :rfc:`3339` format .. code-block:: http :caption: Response Example @@ -127,24 +130,24 @@ Summary Stats "deliveryServiceName": "all", "statName": "daily_maxgbps", "statValue": 5, - "summaryTime": "2019-11-19T00:04:06Z", - "statDate": "2019-11-18T00:00:00Z" + "summaryTime": "2019-11-19 00:04:06+00", + "statDate": "2019-11-19 }, { "cdnName": "CDN-in-a-Box", "deliveryServiceName": "all", "statName": "daily_maxgbps", "statValue": 3, - "summaryTime": "2019-11-18T00:04:06Z", - "statDate": "2019-11-17T00:00:00Z" + "summaryTime": "2019-11-18 00:04:06+00", + "statDate": "2019-11-18" }, { "cdnName": "CDN-in-a-Box", "deliveryServiceName": "all", "statName": "daily_bytesserved", "statValue": 1000, - "summaryTime": "2019-11-19T00:04:06Z", - "statDate": "2019-11-18T00:00:00Z" + "summaryTime": "2019-11-19 00:04:06+00", + "statDate": "2019-11-19" } ]} @@ -170,6 +173,6 @@ Last Updated Summary Stat { "response": { - "summaryTime": "2019-11-19T00:04:06Z" + "summaryTime": "2019-11-19 00:04:06+00" } } \ No newline at end of file diff --git a/lib/go-tc/stats_summary.go b/lib/go-tc/stats_summary.go index c82db4d06f..74075c42fd 100644 --- a/lib/go-tc/stats_summary.go +++ b/lib/go-tc/stats_summary.go @@ -46,7 +46,8 @@ type StatsSummary struct { func (ss *StatsSummary) UnmarshalJSON(data []byte) error { type Alias StatsSummary resp := struct { - StatDate string `json:"statDate"` + SummaryTime string `json:"summaryTime"` + StatDate string `json:"statDate"` *Alias }{ Alias: (*Alias)(ss), @@ -59,18 +60,25 @@ func (ss *StatsSummary) UnmarshalJSON(data []byte) error { if err != nil { return err } - return nil + ss.SummaryTime, err = time.Parse(time.RFC3339, resp.SummaryTime) + if err == nil { + return nil + } + ss.SummaryTime, err = time.Parse(TimeLayout, resp.SummaryTime) + return err } // UnmarshalJSON Customized Marshal to force date format on statDate func (ss StatsSummary) MarshalJSON() ([]byte, error) { type Alias StatsSummary return json.Marshal(&struct { - StatDate string `json:"statDate"` + StatDate string `json:"statDate"` + SummaryTime string `json:"summaryTime"` Alias }{ - StatDate: ss.StatDate.Format(dateFormat), - Alias: (Alias)(ss), + SummaryTime: ss.SummaryTime.Format(TimeLayout), + StatDate: ss.StatDate.Format(dateFormat), + Alias: (Alias)(ss), }) } @@ -79,6 +87,14 @@ type StatsSummaryLastUpdated struct { SummaryTime time.Time `json:"summaryTime" db:"summary_time"` } +func (ss StatsSummaryLastUpdated) MarshalJSON() ([]byte, error) { + return json.Marshal(&struct { + SummaryTime string `json:"summaryTime"` + }{ + SummaryTime: ss.SummaryTime.Format(TimeLayout), + }) +} + // StatsSummaryLastUpdatedResponse ... type StatsSummaryLastUpdatedResponse struct { Response StatsSummaryLastUpdated `json:"response"` diff --git a/traffic_ops/client/stats_summary.go b/traffic_ops/client/stats_summary.go index ef965a75c1..ba755208ff 100644 --- a/traffic_ops/client/stats_summary.go +++ b/traffic_ops/client/stats_summary.go @@ -45,10 +45,7 @@ func (to *Session) GetSummaryStats(cdn, deliveryService, statName *string) (tc.S url = fmt.Sprintf("%s?%s", basePathSummaryStats, param.Encode()) } reqInf, err := get(to, url, &resp) - if err != nil { - return resp, reqInf, err - } - return resp, reqInf, nil + return resp, reqInf, err } // GetSummaryStatsLastUpdated time of the last summary for a given stat @@ -63,10 +60,7 @@ func (to *Session) GetSummaryStatsLastUpdated(statName *string) (tc.StatsSummary url := fmt.Sprintf("%s?%s", basePathSummaryStats, param.Encode()) reqInf, err := get(to, url, &resp) - if err != nil { - return resp, reqInf, err - } - return resp, reqInf, nil + return resp, reqInf, err } // CreateSummaryStats creates a stats summary diff --git a/traffic_ops/testing/api/v14/stats_summary_test.go b/traffic_ops/testing/api/v14/stats_summary_test.go index 547af03b4e..40083a16bd 100644 --- a/traffic_ops/testing/api/v14/stats_summary_test.go +++ b/traffic_ops/testing/api/v14/stats_summary_test.go @@ -32,13 +32,11 @@ var ( const dateFormat = "2006-01-02" func TestStatsSummary(t *testing.T) { - WithObjs(t, []TCObj{}, func() { - testStatsSummaries = []tc.StatsSummary{} - latestTime = time.Now().Truncate(time.Second).UTC() - CreateTestStatsSummaries(t) - GetTestStatsSummaries(t) - GetTestStatsSummariesLastUpdated(t) - }) + testStatsSummaries = []tc.StatsSummary{} + latestTime = time.Now().Truncate(time.Second).UTC() + CreateTestStatsSummaries(t) + GetTestStatsSummaries(t) + GetTestStatsSummariesLastUpdated(t) } func CreateTestStatsSummaries(t *testing.T) { @@ -116,7 +114,7 @@ func GetTestStatsSummaries(t *testing.T) { t.Run(tc.description, func(t *testing.T) { tsr, _, err := TOSession.GetSummaryStats(tc.cdn, tc.ds, tc.stat) if err != nil { - t.Fatalf("received unexpectant error %v on GET to stats_summary", err) + t.Fatalf("received unexpected error %v on GET to stats_summary", err) } if len(tc.expectedStatsSummaries) == 0 && len(tsr.Response) != 0 { t.Fatalf("expected to recieve no stats summaries but received %v", len(tsr.Response)) @@ -130,7 +128,7 @@ func GetTestStatsSummaries(t *testing.T) { } } if !found { - t.Fatalf("expected to find stat %v in stats summary response", ess.StatName) + t.Errorf("expected to find stat %v in stats summary response", ess.StatName) } } }) diff --git a/traffic_ops/testing/api/v14/tc-fixtures.json b/traffic_ops/testing/api/v14/tc-fixtures.json index ba74b365ef..d9b2da5591 100644 --- a/traffic_ops/testing/api/v14/tc-fixtures.json +++ b/traffic_ops/testing/api/v14/tc-fixtures.json @@ -2534,7 +2534,7 @@ "ttl": 2.1 } ], - "stats_summaries": [ + "statsSummaries": [ { "cdnName": "cdn1", "deliveryServiceName": "all", diff --git a/traffic_ops/testing/api/v14/traffic_control_test.go b/traffic_ops/testing/api/v14/traffic_control_test.go index 8da742aabf..806310f9cd 100644 --- a/traffic_ops/testing/api/v14/traffic_control_test.go +++ b/traffic_ops/testing/api/v14/traffic_control_test.go @@ -45,7 +45,7 @@ type TrafficControl struct { ServerCapabilities []tc.ServerCapability `json:"serverCapabilities"` Statuses []tc.StatusNullable `json:"statuses"` StaticDNSEntries []tc.StaticDNSEntry `json:"staticdnsentries"` - StatsSummaries []tc.StatsSummary `json:"stats_summaries"` + StatsSummaries []tc.StatsSummary `json:"statsSummaries"` Tenants []tc.Tenant `json:"tenants"` TOExtensions []tc.TOExtensionNullable `json:"to_extensions"` Types []tc.Type `json:"types"` From da27ab5122a00da50bfc9b4e5645121f039ce92b Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Thu, 5 Dec 2019 17:46:37 -0700 Subject: [PATCH 07/11] Add POST Summary stats --- docs/source/api/stats_summary.rst | 78 ++++++++++++++++++- lib/go-tc/stats_summary.go | 72 ++++++++++++++--- traffic_ops/client/stats_summary.go | 2 +- .../testing/api/v14/stats_summary_test.go | 8 +- .../traffic_ops_golang/routing/routes.go | 1 + .../trafficstats/stats_summary.go | 68 +++++++++++++++- traffic_stats/traffic_stats.go | 16 ++-- 7 files changed, 219 insertions(+), 26 deletions(-) diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst index 8bcd4497ad..379b2f1d65 100644 --- a/docs/source/api/stats_summary.rst +++ b/docs/source/api/stats_summary.rst @@ -100,6 +100,9 @@ Summary Stats """"""""""""" :cdnName: The CDN name for which the summary stat was taken for + + .. note:: If the ``cdn`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN + :deliveryServiceName: The :term:`Delivery Service` display name for which the summary stat was taken for .. note:: If the ``deliveryServiceName`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN @@ -175,4 +178,77 @@ Last Updated Summary Stat { "summaryTime": "2019-11-19 00:04:06+00" } - } \ No newline at end of file + } + +``POST`` +======== +.. versionadded:: 1.4 + +Post a stats summary for a given stat. + +:Auth. Required: Yes +:Roles Required: None +:Response Type: Object + +Request Structure +----------------- +:cdnName: The CDN name for which the summary stat was taken for + + .. note:: If the ``cdn`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN + +:deliveryServiceName: The :term:`Delivery Service` display name for which the summary stat was taken for + + .. note:: If the ``deliveryServiceName`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN + +:statName: Stat name summary stat represents +:statValue: Summary stat value +:summaryTime: Timestamp of summary, in an ISO-like format +:statDate: Date stat was taken, in :rfc:`3339` format + + .. note:: All fields are required besides ``cdnName`` and ``deliveryServiceName`` which will default to ``all`` if not given + +.. code-block:: http + :caption: Request Example + + POST /api/1.4/stats_summary HTTP/1.1 + Host: trafficops.infra.ciab.test + User-Agent: curl/7.47.0 + Accept: */* + Cookie: mojolicious=... + Content-Length: 113 + Content-Type: application/json + + { + "cdnName": "CDN-in-a-Box", + "deliveryServiceName": "all", + "statName": "daily_maxgbps", + "statValue": 10, + "summaryTime": "2019-12-05 00:03:57+00", + "statDate": "2019-12-05" + } + +Response Structure +------------------ +.. code-block:: http + :caption: Response Example + + HTTP/1.1 200 OK + Access-Control-Allow-Credentials: true + Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Set-Cookie, Cookie + Access-Control-Allow-Methods: POST,GET,OPTIONS,PUT,DELETE + Access-Control-Allow-Origin: * + Content-Type: application/json + Set-Cookie: mojolicious=...; Path=/; Expires=Mon, 18 Nov 2019 17:40:54 GMT; Max-Age=3600; HttpOnly + Whole-Content-Sha512: ezxk+iP7o7KE7zpWmGc0j8nz5k+1wAzY0HiNiA2xswTQrt+N+6CgQqUV2r9G1HAsPNr0HF2PhYs/Xr7DrYOw0A== + X-Server-Name: traffic_ops_golang/ + Date: Thu, 06 Dec 2018 02:14:45 GMT + Content-Length: 97 + + { + "alerts": [ + { + "text": "Stats Summary was successfully created", + "level": "success" + } + ] + } diff --git a/lib/go-tc/stats_summary.go b/lib/go-tc/stats_summary.go index 74075c42fd..fd41a1b2cb 100644 --- a/lib/go-tc/stats_summary.go +++ b/lib/go-tc/stats_summary.go @@ -1,8 +1,14 @@ package tc import ( + "database/sql" "encoding/json" + "errors" "time" + + "github.com/apache/trafficcontrol/lib/go-tc/tovalidate" + "github.com/apache/trafficcontrol/lib/go-util" + validation "github.com/go-ozzo/ozzo-validation" ) /* @@ -33,15 +39,22 @@ type StatsSummaryResponse struct { // StatsSummary ... type StatsSummary struct { - ID int `json:"-" db:"id"` - CDNName string `json:"cdnName" db:"cdn_name"` - DeliveryService string `json:"deliveryServiceName" db:"deliveryservice_name"` - StatName string `json:"statName" db:"stat_name"` - StatValue float64 `json:"statValue" db:"stat_value"` + CDNName *string `json:"cdnName" db:"cdn_name"` + DeliveryService *string `json:"deliveryServiceName" db:"deliveryservice_name"` + StatName *string `json:"statName" db:"stat_name"` + StatValue *float64 `json:"statValue" db:"stat_value"` SummaryTime time.Time `json:"summaryTime" db:"summary_time"` StatDate time.Time `json:"statDate" db:"stat_date"` } +func (ss StatsSummary) Validate(tx *sql.Tx) error { + errs := tovalidate.ToErrors(validation.Errors{ + "statName": validation.Validate(ss.StatName, validation.Required), + "statValue": validation.Validate(ss.StatValue, validation.Required), + }) + return util.JoinErrs(errs) +} + // UnmarshalJSON Customized Unmarshal to force date format on statDate func (ss *StatsSummary) UnmarshalJSON(data []byte) error { type Alias StatsSummary @@ -56,16 +69,39 @@ func (ss *StatsSummary) UnmarshalJSON(data []byte) error { if err != nil { return err } - ss.StatDate, err = time.Parse(dateFormat, resp.StatDate) + + ss.StatDate, err = parseStatDate(resp.StatDate) if err != nil { - return err + return errors.New("invalid timestamp given for statDate") + } + + ss.SummaryTime, err = parseSummaryTime(resp.SummaryTime) + if err != nil { + return errors.New("invalid timestamp given for summaryTime") } - ss.SummaryTime, err = time.Parse(time.RFC3339, resp.SummaryTime) + return nil +} + +func parseStatDate(sd string) (time.Time, error) { + rt, err := time.Parse(time.RFC3339, sd) if err == nil { - return nil + return rt, err } - ss.SummaryTime, err = time.Parse(TimeLayout, resp.SummaryTime) - return err + rt, err = time.Parse(TimeLayout, sd) + if err == nil { + return rt, err + } + return time.Parse(dateFormat, sd) +} + +func parseSummaryTime(st string) (time.Time, error) { + // var rt time.Time + // var err error + rt, err := time.Parse(time.RFC3339, st) + if err == nil { + return rt, err + } + return time.Parse(TimeLayout, st) } // UnmarshalJSON Customized Marshal to force date format on statDate @@ -95,6 +131,20 @@ func (ss StatsSummaryLastUpdated) MarshalJSON() ([]byte, error) { }) } +// UnmarshalJSON Customized Unmarshal to force date format on summaryTime +func (ssl *StatsSummaryLastUpdated) UnmarshalJSON(data []byte) error { + resp := struct { + SummaryTime string `json:"summaryTime"` + }{} + err := json.Unmarshal(data, &resp) + if err != nil { + return err + } + ssl.SummaryTime, err = time.Parse(TimeLayout, resp.SummaryTime) + return err + +} + // StatsSummaryLastUpdatedResponse ... type StatsSummaryLastUpdatedResponse struct { Response StatsSummaryLastUpdated `json:"response"` diff --git a/traffic_ops/client/stats_summary.go b/traffic_ops/client/stats_summary.go index ba755208ff..acfe134408 100644 --- a/traffic_ops/client/stats_summary.go +++ b/traffic_ops/client/stats_summary.go @@ -72,6 +72,6 @@ func (to *Session) CreateSummaryStats(statsSummary tc.StatsSummary) (tc.Alerts, if err != nil { return tc.Alerts{}, reqInf, err } - reqInf, err = post(to, fmt.Sprintf("%s/%s", basePathSummaryStats, "create"), reqBody, &alerts) + reqInf, err = post(to, basePathSummaryStats, reqBody, &alerts) return alerts, reqInf, err } diff --git a/traffic_ops/testing/api/v14/stats_summary_test.go b/traffic_ops/testing/api/v14/stats_summary_test.go index 40083a16bd..20ceda6c97 100644 --- a/traffic_ops/testing/api/v14/stats_summary_test.go +++ b/traffic_ops/testing/api/v14/stats_summary_test.go @@ -91,7 +91,7 @@ func GetTestStatsSummaries(t *testing.T) { expectedStatsSummaries: func() []tc.StatsSummary { statsSummaries := []tc.StatsSummary{} for _, ss := range testStatsSummaries { - if ss.StatName == "daily_bytesserved" { + if *ss.StatName == "daily_bytesserved" { statsSummaries = append(statsSummaries, ss) } } @@ -122,13 +122,13 @@ func GetTestStatsSummaries(t *testing.T) { for _, ess := range tc.expectedStatsSummaries { found := false for _, ss := range tsr.Response { - if ess == ss { + if *ess.StatName == *ss.StatName && ess.SummaryTime.Equal(ss.SummaryTime) && ess.StatDate.Equal(ss.StatDate) { found = true break } } if !found { - t.Errorf("expected to find stat %v in stats summary response", ess.StatName) + t.Errorf("expected to find stat %v in stats summary response", *ess.StatName) } } }) @@ -158,7 +158,7 @@ func GetTestStatsSummariesLastUpdated(t *testing.T) { for _, ss := range testStatsSummaries { testCases = append(testCases, testCase{ description: fmt.Sprintf("latest updated timestamp for - %v", ss.StatName), - stat: util.StrPtr(ss.StatName), + stat: ss.StatName, errExpected: false, expectedTimestamp: ss.SummaryTime, }) diff --git a/traffic_ops/traffic_ops_golang/routing/routes.go b/traffic_ops/traffic_ops_golang/routing/routes.go index d585214b2f..3a1d866ad7 100644 --- a/traffic_ops/traffic_ops_golang/routing/routes.go +++ b/traffic_ops/traffic_ops_golang/routing/routes.go @@ -556,6 +556,7 @@ func Routes(d ServerData) ([]Route, []RawRoute, http.Handler, error) { // Stats Summary {1.1, http.MethodGet, `stats_summary/?(\.json)?$`, trafficstats.GetStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil, 380498598, perlBypass}, + {1.4, http.MethodPost, `stats_summary/?(\.json)?$`, trafficstats.CreateStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil, 380491598, noPerlBypass}, //Pattern based consistent hashing endpoint {1.4, http.MethodPost, `consistenthash/?$`, consistenthash.Post, auth.PrivLevelReadOnly, Authenticated, nil, 1960755076, noPerlBypass}, diff --git a/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go b/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go index 61f17a1f10..421bffc230 100644 --- a/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go +++ b/traffic_ops/traffic_ops_golang/trafficstats/stats_summary.go @@ -114,9 +114,56 @@ func queryStatsSummary(tx *sqlx.Tx, q string, queryValues map[string]interface{} return statsSummaries, nil } +// CreateStatsSummary handler for creating stats summaries +func CreateStatsSummary(w http.ResponseWriter, r *http.Request) { + inf, userErr, sysErr, errCode := api.NewInfo(r, []string{}, []string{}) + if userErr != nil || sysErr != nil { + api.HandleErr(w, r, inf.Tx.Tx, errCode, userErr, sysErr) + return + } + defer inf.Close() + + ss := tc.StatsSummary{} + + if err := api.Parse(r.Body, inf.Tx.Tx, &ss); err != nil { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusBadRequest, err, nil) + return + } + + // CDN Name and Delivery service name are defaulted to "all" if not defined + if ss.CDNName == nil || len(*ss.CDNName) == 0 { + ss.CDNName = util.StrPtr("all") + } + + if ss.DeliveryService == nil || len(*ss.DeliveryService) == 0 { + ss.DeliveryService = util.StrPtr("all") + } + + id := -1 + rows, err := inf.Tx.NamedQuery(insertQuery(), &ss) + if err != nil { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, fmt.Errorf("inserting stats summary: %v", err)) + return + } + for rows.Next() { + if err := rows.Scan(&id); err != nil { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, fmt.Errorf("scanning created stats summary id: %v", err)) + return + } + } + if id == -1 { + api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, fmt.Errorf("sstats summary id: %v", err)) + return + } + + successMsg := "Stats Summary was successfully created" + logMsg := fmt.Sprintf("STATSSUMMARY: %v, ID: %v, ACTION: create stats_summary", *ss.StatName, id) + api.CreateChangeLogRawTx(api.ApiChange, logMsg, inf.User, inf.Tx.Tx) + api.WriteRespAlert(w, r, tc.SuccessLevel, successMsg) +} + func selectQuery() string { return `SELECT -id, cdn_name, deliveryservice_name, stat_name, @@ -125,3 +172,22 @@ summary_time, stat_date FROM stats_summary` } + +func insertQuery() string { + return ` +INSERT INTO stats_summary ( + cdn_name, + deliveryservice_name, + stat_name, + stat_value, + summary_time, + stat_date) +VALUES ( + :cdn_name, + :deliveryservice_name, + :stat_name, + :stat_value, + :summary_time, + :stat_date) RETURNING id +` +} diff --git a/traffic_stats/traffic_stats.go b/traffic_stats/traffic_stats.go index 77139a20a8..9ec38a8a27 100644 --- a/traffic_stats/traffic_stats.go +++ b/traffic_stats/traffic_stats.go @@ -315,10 +315,10 @@ func calcDailyMaxGbps(client influx.Client, bp influx.BatchPoints, startTime tim statTime, _ := time.Parse(time.RFC3339, t) log.Infof("max gbps for cdn %v = %v", cdn, value) var statsSummary tc.StatsSummary - statsSummary.CDNName = cdn - statsSummary.DeliveryService = "all" - statsSummary.StatName = "daily_maxgbps" - statsSummary.StatValue = value + statsSummary.CDNName = util.StrPtr(cdn) + statsSummary.DeliveryService = util.StrPtr("all") + statsSummary.StatName = util.StrPtr("daily_maxgbps") + statsSummary.StatValue = util.FloatPtr(value) statsSummary.SummaryTime = time.Now() statsSummary.StatDate = statTime go writeSummaryStats(config, statsSummary) @@ -375,10 +375,10 @@ func calcDailyBytesServed(client influx.Client, bp influx.BatchPoints, startTime log.Infof("TBytes served for cdn %v = %v", cdn, bytesServedTB) //write to Traffic Ops var statsSummary tc.StatsSummary - statsSummary.CDNName = cdn - statsSummary.DeliveryService = "all" - statsSummary.StatName = "daily_bytesserved" - statsSummary.StatValue = bytesServedTB + statsSummary.CDNName = util.StrPtr(cdn) + statsSummary.DeliveryService = util.StrPtr("all") + statsSummary.StatName = util.StrPtr("daily_bytesserved") + statsSummary.StatValue = util.FloatPtr(bytesServedTB) statsSummary.SummaryTime = time.Now() statsSummary.StatDate = startTime go writeSummaryStats(config, statsSummary) From cf06e369878e797ee17c6d968efc48eab200faf8 Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Mon, 6 Jan 2020 08:44:03 -0700 Subject: [PATCH 08/11] Move stats_summary POST to 1.5 --- CHANGELOG.md | 2 ++ docs/source/api/stats_summary.rst | 28 +++++++++---------- lib/go-tc/stats_summary.go | 20 ++++--------- traffic_ops/client/endpoints.go | 2 +- .../testing/api/v14/stats_summary_test.go | 3 +- .../traffic_ops_golang/routing/routes.go | 2 +- 6 files changed, 24 insertions(+), 33 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8445c01cb3..80a8964608 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). ## [unreleased] ### Added +- Traffic Ops Endpoints + - /api/1.5/stats_summary `POST` ### Changed diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst index c3ad0bc050..9c2067e148 100644 --- a/docs/source/api/stats_summary.rst +++ b/docs/source/api/stats_summary.rst @@ -179,7 +179,7 @@ Last Updated Summary Stat }} ``POST`` -.. versionadded:: 1.4 +.. versionadded:: 1.4.1 Post a stats summary for a given stat. @@ -207,7 +207,7 @@ Request Structure .. code-block:: http :caption: Request Example - POST /api/1.4/stats_summary HTTP/1.1 + POST /api/1.5/stats_summary HTTP/1.1 Host: trafficops.infra.ciab.test User-Agent: curl/7.47.0 Accept: */* @@ -216,12 +216,12 @@ Request Structure Content-Type: application/json { - "cdnName": "CDN-in-a-Box", - "deliveryServiceName": "all", - "statName": "daily_maxgbps", - "statValue": 10, - "summaryTime": "2019-12-05 00:03:57+00", - "statDate": "2019-12-05" + "cdnName": "CDN-in-a-Box", + "deliveryServiceName": "all", + "statName": "daily_maxgbps", + "statValue": 10, + "summaryTime": "2019-12-05 00:03:57+00", + "statDate": "2019-12-05" } Response Structure @@ -241,11 +241,9 @@ Response Structure Date: Thu, 06 Dec 2018 02:14:45 GMT Content-Length: 97 - { - "alerts": [ - { - "text": "Stats Summary was successfully created", - "level": "success" - } - ] + { "alerts": [ + { + "text": "Stats Summary was successfully created", + "level": "success" + }] } diff --git a/lib/go-tc/stats_summary.go b/lib/go-tc/stats_summary.go index e4c82bb64d..d43ff23987 100644 --- a/lib/go-tc/stats_summary.go +++ b/lib/go-tc/stats_summary.go @@ -70,38 +70,30 @@ func (ss *StatsSummary) UnmarshalJSON(data []byte) error { return err } if resp.StatDate != nil { - statDate, err := parseStatDate(*resp.StatDate) + statDate, err := parseTime(*resp.StatDate) if err != nil { return errors.New("invalid timestamp given for statDate") } ss.StatDate = &statDate } - ss.SummaryTime, err = parseSummaryTime(resp.SummaryTime) + ss.SummaryTime, err = parseTime(resp.SummaryTime) if err != nil { return errors.New("invalid timestamp given for summaryTime") } return nil } -func parseStatDate(sd string) (time.Time, error) { - rt, err := time.Parse(time.RFC3339, sd) +func parseTime(ts string) (time.Time, error) { + rt, err := time.Parse(time.RFC3339, ts) if err == nil { return rt, err } - rt, err = time.Parse(TimeLayout, sd) + rt, err = time.Parse(TimeLayout, ts) if err == nil { return rt, err } - return time.Parse(dateFormat, sd) -} - -func parseSummaryTime(st string) (time.Time, error) { - rt, err := time.Parse(time.RFC3339, st) - if err == nil { - return rt, err - } - return time.Parse(TimeLayout, st) + return time.Parse(dateFormat, ts) } // UnmarshalJSON Customized Marshal to force date format on statDate diff --git a/traffic_ops/client/endpoints.go b/traffic_ops/client/endpoints.go index bd71a01c2b..2df163e049 100644 --- a/traffic_ops/client/endpoints.go +++ b/traffic_ops/client/endpoints.go @@ -15,4 +15,4 @@ package client -const apiBase = "/api/1.4" +const apiBase = "/api/1.5" diff --git a/traffic_ops/testing/api/v14/stats_summary_test.go b/traffic_ops/testing/api/v14/stats_summary_test.go index 03395ddd2f..b7b74e7bd2 100644 --- a/traffic_ops/testing/api/v14/stats_summary_test.go +++ b/traffic_ops/testing/api/v14/stats_summary_test.go @@ -150,7 +150,7 @@ func GetTestStatsSummariesLastUpdated(t *testing.T) { } for _, ss := range testStatsSummaries { testCases = append(testCases, testCase{ - description: fmt.Sprintf("latest updated timestamp for - %v", ss.StatName), + description: fmt.Sprintf("latest updated timestamp for - %v", *ss.StatName), stat: ss.StatName, errExpected: false, expectedTimestamp: ss.SummaryTime, @@ -175,5 +175,4 @@ func GetTestStatsSummariesLastUpdated(t *testing.T) { } }) } - } diff --git a/traffic_ops/traffic_ops_golang/routing/routes.go b/traffic_ops/traffic_ops_golang/routing/routes.go index 3a25bf867c..916d4d14d7 100644 --- a/traffic_ops/traffic_ops_golang/routing/routes.go +++ b/traffic_ops/traffic_ops_golang/routing/routes.go @@ -573,7 +573,7 @@ func Routes(d ServerData) ([]Route, []RawRoute, http.Handler, error) { // Stats Summary {1.1, http.MethodGet, `stats_summary/?(\.json)?$`, trafficstats.GetStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil, 380498598, perlBypass}, - {1.4, http.MethodPost, `stats_summary/?(\.json)?$`, trafficstats.CreateStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil, 380491598, noPerlBypass}, + {1.5, http.MethodPost, `stats_summary/?(\.json)?$`, trafficstats.CreateStatsSummary, auth.PrivLevelReadOnly, Authenticated, nil, 380491598, noPerlBypass}, //Pattern based consistent hashing endpoint {1.4, http.MethodPost, `consistenthash/?$`, consistenthash.Post, auth.PrivLevelReadOnly, Authenticated, nil, 1960755076, noPerlBypass}, From df86ac90561c5f95344b1e91eb12523d90fc340f Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Fri, 10 Jan 2020 09:05:03 -0700 Subject: [PATCH 09/11] Fix docs issues --- docs/source/api/stats_summary.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst index 9c2067e148..f3680aed6e 100644 --- a/docs/source/api/stats_summary.rst +++ b/docs/source/api/stats_summary.rst @@ -101,7 +101,7 @@ Summary Stats :cdnName: The CDN name for which the summary stat was taken for - .. note:: If the ``cdn`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN + .. note:: If the ``cdn`` is equal to ``all`` it represents summary_stats across all delivery services across all CDNs :deliveryServiceName: The :term:`Delivery Service` display name for which the summary stat was taken for @@ -179,7 +179,9 @@ Last Updated Summary Stat }} ``POST`` -.. versionadded:: 1.4.1 +======== + +.. versionadded:: 1.5 Post a stats summary for a given stat. @@ -202,7 +204,7 @@ Request Structure :summaryTime: Timestamp of summary, in an ISO-like format :statDate: Date stat was taken, in :rfc:`3339` format - .. note:: All fields are required besides ``cdnName`` and ``deliveryServiceName`` which will default to ``all`` if not given +.. note:: ``statName``, ``statValue`` and ``summaryTime`` are required. If ``cdnName`` and ``deliveryServiceName`` are not given they will default to ``all``. .. code-block:: http :caption: Request Example From 4ece636588b80d0b75349c216987dc2f52f22052 Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Tue, 14 Jan 2020 09:19:50 -0700 Subject: [PATCH 10/11] Actually adress comments for docs --- docs/source/api/stats_summary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/api/stats_summary.rst b/docs/source/api/stats_summary.rst index f3680aed6e..5de0568901 100644 --- a/docs/source/api/stats_summary.rst +++ b/docs/source/api/stats_summary.rst @@ -193,7 +193,7 @@ Request Structure ----------------- :cdnName: The CDN name for which the summary stat was taken for - .. note:: If the ``cdn`` is equal to ``all`` it represents summary_stats across all delivery services within the given CDN + .. note:: If the ``cdn`` is equal to ``all`` it represents summary_stats across all delivery services across all CDNs :deliveryServiceName: The :term:`Delivery Service` display name for which the summary stat was taken for From 171d02950ef1ebc4fde8506e1ccf9cc3d0e0f8b4 Mon Sep 17 00:00:00 2001 From: Michael Hoppal Date: Wed, 29 Jan 2020 09:05:12 -0700 Subject: [PATCH 11/11] Update changelog --- CHANGELOG.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ec4e7255e2..8595bc1b48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,14 +5,13 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). ## [unreleased] ### Added -- Traffic Ops Endpoints - - /api/1.5/stats_summary `POST` - Added a boolean to delivery service in Traffic Portal and Traffic Ops to enable EDNS0 client subnet at the delivery service level and include it in the cr-config. - Updated Traffic Router to read new EDSN0 client subnet field and route accordingly only for enabled delivery services. When enabled and a subnet is present in the request, the subnet appears in the `chi` field and the resolver address is in the `rhi` field. - Added an optimistic quorum feature to Traffic Monitor to prevent false negative states from propagating to downstream components in the event of network isolation. - Traffic Ops Golang Endpoints - /api/1.1/cachegroupparameters/{{cachegroupID}}/{{parameterID}} `(DELETE)` - /api/1.5/to_extensions `(POST)` + - /api/1.5/stats_summary `(POST)` ### Changed