After reading "The ASA's statement on pvalues: context, process, and purpose", and some other related references, here are some excerpts and notes I took on pvalue and nullhypothesis significance testing.

American Statistical Association (ASA) has stated the following five principles about pvalues and null hypothesis significance testing:
 "Pvalues can indicate how incompatible the data are with a specified statistical model."
 "Pvalues do not measure the probability that the studied
hypothesis is true, or the probability that the data were produced
by random chance alone."
 " … It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself."
 "Scientific conclusions and business or policy decisions should
not be based only on whether a pvalue passes a specific
threshold."
 "… Practices that reduce data analysis or scientific inference to mechanical “brightline” rules (such as “p < 0.05”) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decisionmaking. …"
 "Proper inference requires full reporting and transparency."
 "A pvalue, or statistical significance, does not measure the
size of an effect or the importance of a result."
 "… Smaller pvalues do not necessarily imply the presence of larger or more important effects, and larger pvalues do not imply a lack of importance or even lack of effect. Any effect, no matter how tiny, can produce a small pvalue if the sample size or measurement precision is high enough, and large effects may produce unimpressive pvalues if the sample size is small or measurements are imprecise. …"

Null hypothesis is usually a hypothesis that assumes that observed data and its distribution is a result of random chances rather than that of effects caused by some intrinsic mechanisms. It is usually what is to disapprove or to reject in order to establish evidence to or belief in that there is some real effect due to underlying intrinsic mechanism. In turn, the details of the statistical model used in this evaluation can be used to make quantitative estimations on properties of the underlying mechanism.

The pvalue is the probability that one has falsely rejected the null hypothesis.
 The smaller is, the smaller the chance is that one has falsely rejected the null hypothesis.
 Being able to reject or not being able to reject the null hypothesis may tells one if the observed data suggests that there is an effect, however, it does not tell one how much an effect there is and if the effect is true. See effect size.
 "a pvalue near 0.05 taken by itself offers only weak evidence against the null hypothesis".
 UK statistician and geneticist Sir Ronald Fisher introduced the pvalue in the 1920s. "The pvalue was never meant to be used the way it's used today."

As ASA pvalue principle No. 3 states, the decision to reject the null hypothesis should not be based solely on if pvalue passes a "brightline" threshold. Rather, in order to reject the null hypothesis, one must make a subjective judgment involving the degree of risk acceptable for being wrong. The degree of risk of being wrong may be specified in terms of confidence levels which characterizes the sampling variability.

Alternative ways used for referring to data cherrypicking include data dredging, significance chasing, significance questing, selective inference, phacking, snooping, fishing, and doubledipping.

"The difference between statistically significant and statistically insignificant is not, itself, statistically significant."

"According to one widely used calculation [1], a pvalue of 0.01 corresponds to a falsealarm probability of at least 11%, depending on the underlying probability that there is a true effect; a pvalue of 0.05 raises that chance to at least 29%." See the following figure:
?pvalue and probable cause.png
Some related concepts

The standard score, or zscore is the deviation from the mean in units of standard deviation. A small pvalue corresponds to a large positive zscore.

 Magnitude  How big is the effect? Large effects are more compelling than small ones.
 Articulation  How specific is it? Precise statements are more compelling than imprecise ones.
 Generality  How generally does it apply?
 Interestingness  interesting effects are those that "have the potential, through empirical analysis, to change what people believe about an important issue".
 Credibility  Credible claims are more compelling than incredible ones. The researcher must show that the claims made are credible.
References

"The problem with pvalues: how significant are they, really?", phys.org Science News Wire, 2013, http://phys.org/wirenews/145707973/theproblemwithpvalueshowsignificantaretheyreally.html

Regina Nuzzo, "Scientific method: statistical errors," 2014, http://folk.ntnu.no/slyderse/Nuzzo%20and%20Editorial%20%20pvalues.pdf

Tom Siegfried, "Odds Are, It's Wrong  Science fails to face the shortcomings of statistics," 2010, https://www.sciencenews.org/article/oddsareitswrong

Gelman, A., and Loken, E., "The Statistical Crisis in Science," American Scientist, 102., 2014, http://www.americanscientist.org/issues/feature/2014/6/thestatisticalcrisisinscience

"The vast majority of statistical analysis is not performed by statisticians," simplystatistics.org, 2013, http://simplystatistics.org/2013/06/14/thevastmajorityofstatisticalanalysisisnotperformedbystatisticians/

"On the scalability of statistical procedures: why the pvalue bashers just don't get it," simplystatistics.org, 2014, http://simplystatistics.org/2014/02/14/onthescalabilityofstatisticalprocedureswhythepvaluebashersjustdontgetit/

Andrew Gelmana and Hal Sterna, The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant, The American Statistician, Volume 60, Issue 4, 2006, http://www.tandfonline.com/doi/abs/10.1198/000313006X152649

Goodman, "Of PValues and Bayes: A Modest Proposal," S. N. Epidemiology 12, 295–297 (2001), http://journals.lww.com/epidem/fulltext/2001/05000/of_p_values_and_bayes__a_modest_proposal.6.aspx ↩