Skip to content

HDDS-2035 Implement datanode level CLI to reveal pipeline relation.#348

Merged
ChenSammi merged 2 commits intoapache:HDDS-1564from
timmylicheng:HDDS-2035
Dec 18, 2019
Merged

HDDS-2035 Implement datanode level CLI to reveal pipeline relation.#348
ChenSammi merged 2 commits intoapache:HDDS-1564from
timmylicheng:HDDS-2035

Conversation

@timmylicheng
Copy link
Copy Markdown
Contributor

@timmylicheng timmylicheng commented Dec 12, 2019

What changes were proposed in this pull request?

#HDDS-2035 Implement datanode level CLI to reveal pipeline info

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-2035

(Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)

Please replace this section with the link to the Apache JIRA)

How was this patch tested?

bash-4.2$ ozone scmcli datanode list
Datanode: 7a68a668-3dbe-4366-9fed-271f221c1db0 (172.20.0.2/ozone_datanode_1.ozone_default).
Related pipelines:
cd060313-b724-46cf-a83a-8764ba066ae2/ONE/RATIS/OPEN

(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

adoroszlai
adoroszlai reviewed Dec 13, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @timmylicheng for working on this.

@timmylicheng timmylicheng force-pushed the HDDS-2035 branch 3 times, most recently from 501b642 to 340f5e8 Compare December 17, 2019 03:44
@timmylicheng
Copy link
Copy Markdown
Contributor Author

timmylicheng commented Dec 17, 2019

testCloseContainerEventWithRatis is fixed in #291

@ChenSammi
Copy link
Copy Markdown
Contributor

ChenSammi commented Dec 17, 2019

Command line filter doesn't take effect.

bash-4.2$ ozone scmcli datanode list --id=aaaa

Datanode: a37d1f73-7b4c-459a-a119-cd33c0fd64da (172.18.0.4/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
a1065d21-77a5-4265-80db-d341ac7c3da0/ONE/RATIS/OPEN/Leader

adoroszlai
adoroszlai requested changes Dec 17, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream.filter creates a new stream, which needs to be assigned to the existing stream to let the final forEach see the result of filtering.

@timmylicheng
Copy link
Copy Markdown
Contributor Author

timmylicheng commented Dec 17, 2019

Stream.filter creates a new stream, which needs to be assigned to the existing stream to let the final forEach see the result of filtering.

@adoroszlai I have made the changes you requested.

@timmylicheng
Copy link
Copy Markdown
Contributor Author

timmylicheng commented Dec 17, 2019 •
edited
Loading

Command line filter doesn't take effect.

bash-4.2$ ozone scmcli datanode list --id=aaaa

Datanode: a37d1f73-7b4c-459a-a119-cd33c0fd64da (172.18.0.4/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
a1065d21-77a5-4265-80db-d341ac7c3da0/ONE/RATIS/OPEN/Leader

bash-4.2$ ozone scmcli datanode list
Datanode: 63b02976-3fee-48dc-9f21-b42d7b6e4a40 (172.31.0.3/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
c8455063-62cc-4bb2-a63c-50a9d59a3adf/ONE/RATIS/ALLOCATED/Follower

bash-4.2$ ozone scmcli datanode list --ip 1.1.1.1

@ChenSammi Just updated.

adoroszlai
adoroszlai approved these changes Dec 17, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @timmylicheng for updating the patch. Tried it locally applied to master, works fine.

@ChenSammi
Copy link
Copy Markdown
Contributor

ChenSammi commented Dec 18, 2019

Thanks @timmylicheng for the contribution. Committed to branch.

@ChenSammi ChenSammi merged commit e7315e9 into apache:HDDS-1564 Dec 18, 2019
@timmylicheng timmylicheng deleted the HDDS-2035 branch December 19, 2019 02:25
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Dec 23, 2019
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
8c07782 
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Jan 7, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
3796154 
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 10, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
81ed727 
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 12, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
e720e7a 
…pache#348)
anuengineer pushed a commit that referenced this pull request Feb 19, 2020
@timmylicheng @ChenSammi @xiaoyuyao
HDDS-1564 Multiraft support (#538)
4ba1932 
* HDDS-1577. Add default pipeline placement policy implementation. (#1366)



(cherry picked from commit b640a5f6d53830aee4b9c2a7d17bf57c987962cd)

* HDDS-1571. Create an interface for pipeline placement policy to support network topologies. (#1395)

(cherry picked from commit 753fc6703a39154ed6013e44dbae572391748906)

* HDDS-2089: Add createPipeline CLI. (#1418)

(cherry picked from commit 326b5acd4a63fe46821919322867f5daff30750c)

* HDDS-1569 Support creating multiple pipelines with same datanode. Contributed by Li Cheng. 

This closes #28

* HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN pipeline. (#237)

* Rebase Fix

* HDDS-2650 Fix createPipeline CLI. (#340)

* HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (#348)

* Revert "HDDS-2650 Fix createPipeline CLI. (#340)"

This reverts commit 7c71710.

* HDDS-2650 Fix createPipeline CLI and make it message based. (#370)

* HDDS-1574 Average out pipeline allocation on datanodes and add metrcs/test (#291)

* Resolve rebase conflict.

* HDDS-2756. Handle pipeline creation failure in different way when it exceeds pipeline limit

Closes #401

* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI (#375)

* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI.

* HDDS-2772 Better management for pipeline creation limitation. (#410)

*  HDDS-2913 Update config names and CLI for multi-raft feature. (#462)

* HDDS-2924. Fix Pipeline#nodeIdsHash collision issue. (#478)

* HDDS-2923 Add fall-back protection for rack awareness in pipeline creation. (#516)

* HDDS-3007 Fix CI test failure for TestSCMNodeManager. (#550)

Co-authored-by: Sammi Chen <sammichen@apache.org>
Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adoroszlai adoroszlai adoroszlai approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@timmylicheng @ChenSammi @adoroszlai