Science refers to a systematized body of knowledge. It studies cause and effect relationship and attempts to make generalization in the form of science principles or laws.
Statistics is an art as well as a science, so it’s important to use it as you would any other tool: you have to know what you want to get out of your statistics before you can choose the right way to apply them. Therefore, let’s start with some philosophy.
First, statistics are not magic! If the data aren’t good enough, no amount of statistical manipulation will bring out the results. If your data are exceptional, the main results are often evident even without lots of massaging. Nonetheless, there are plenty of cases in the middle (where only good statistics will work), and it’s appropriate to be rigorous when possible anyway.
Second, in astronomy systematic is often a major issue. An important principle is that hypotheses are tested in bundles. When you make an observation, you aren’t measuring the physical quantities of the system directly. You make implicit assumptions about your knowledge of the instrument, and often need to make explicit assumptions about a basic model, to get the quantities that really interest you. Often there are enormous uncertainties as a result. This means that the statistical significance you quote may not be representative.
Statistics is the science of collecting, organizing, presenting, analyzing, and interpreting numerical data for the purpose of assisting in making a more effective decision.
Art, on the other hand, refers to the skill of handling facts so as to achieve a given objective. It is concerned with ways and means of presenting and handling data making inferences logically and drawing relevant conclusions.
Statistics may thus be viewed both science and art.
